0

I am making an application that will allow user to store and execute simple scripts using Web GUI. Web GUI is ASP.NET MVC and scripts can be of any kind (Python, Perl, AWK, CMD, PowerShell ect. User just needs to provide path to executable). Also there are scripts that do not require executable like XSLT, IronPython ect. Scripts will perform some simple data processing, but it's nature is totally depended on user, so it cannot be wired into application itself.

Question is what is the best approach to store these scripts?

  1. Store all scripts in files and provide script files to interpreters.

It is relatively easy to make solution, but some metadata and relations to other parts of system will be still stored in Db. This also allows to change scripts externally using user's favorite text editor and allows to debug scripts.

But I am concerned if is it safe solution, given that multiple users could be accessing to and executing same files at the same time, since they operate from web GUI.

  1. Store all scripts in Db and create temporary file when execution is required. Provide tmp file to interpreter and remove it once job is done.

  2. Store all scripts in Db and provide them to interpreters via command line (for example python.exe -c ).

Problem with this approach is that I need to heavily escape the script code in order to paste it as command line argument.

Could you please provide const and pros of these approaches?

Many thanks!

P.S.

Security is not a concern for me. At least not now. All scripst are managed by veryfied authneticated users.

  • What if I told you, that the file system is a database? - The question here is not really whatever or not you store them in a database, but one of what kind. I presume you are waging relational databases vs the file system, yet there are other options to consider. I need to know more about the requriements, for example you mention editing the scripts, is this something regular users should do? Also, what data you need to associate with the scripts (in particular, do you need to reference other data you have in a relational database)? – Theraot Jun 7 '17 at 20:33
  • @Theraot Data could be extracted from various sources, for now I will include functionality allowing extraction from local files. But later I want to expand it to SQL and may be web. The way data are extracted is defined again by user with scripts. In general, app will manage many scripts and will allow user to manage and present data by pipelining selected scripts and executing them. I will provide predefined example scripts for data extracting and transformation and I want to allow users to add their own functionality. – Roman Artemov Jun 8 '17 at 5:19
  • It also worth to mention that I need to give to interpreter's process the source code, optional input multi-line data and in some cases additional multi-line data to modify script behavior (for example template for 3rd party library used in script). – Roman Artemov Jun 8 '17 at 6:30
1

An advice: Do not think of your solution as a monolith. Be open to the idea of it being more than one program.


Pros and Cons

Honeslty, I am unsure if these are pros on cons, I leave it to your judgement...

  1. Store all scripts in Db and provide them to interpreters via command line (for example python.exe -c ).
  • The scripts cannot use the file system.
  • Not metaprogramming friendly.
  1. Store all scripts in Db and create temporary file when execution is required. Provide tmp file to interpreter and remove it once job is done.
  • You can allow the scripts to use the file system.
  • Metaprogramming friendly.
  1. Store all scripts in files and provide script files to interpreters.
  • You can allow the scripts to use the file system.
  • Not metaprogramming friendly (if you want to protect scripts from each other).

Errata: It is possible to allow scripts to communicate/interfere with each other in any option. It is easier on option 1.

How safe they are? Well, they are about the same. The measures you need to take to make your system secure has nothing to do with the storage of the scripts, but how you run them... except that doesn't mean what you are thinking.


Security Concerns:

Just running arbitrary third party code is security concern. Two concerns actually:

  1. Do not allow scripts to mess with other scripts
  2. Do not allow scripts to mess with the underlying system

For the first concern, it could appear that taking advantage of a database engine and withholding access to it will prevent a malicious script from messing with other scripts. However, the malicious script can still mess with the rest of the system (which might or might not include tampering with the database engine).

The scripts could cause a lot of damage to your system. Just as an example, one could download malware and configure it run on a scheduled task or on reboot.

In terms of the second concern, the database is not buying you much security.

In addition, passing the code via command line will not make it less harmful.


For that second concern, your first real option is to run the code as an operating system user with low privileges. Besides that, I would suggest virtualization solutions (containers, virtual machines, or other generic sandboxes).

Now, if you consider running your server code under a user with low privileges, you will have to use option 3:

Store all scripts in Db and provide them to interpreters via command line (for example python.exe -c ).

That is, because, if you have right to write files, so will the scripts. Thus, better do not have that right.

Note: You can't really block them from the network with this method.


The database here might be of any kind. In fact, it could be storing the scripts in plain in your file system. Except, you do not pass the script to the interpreter, instead you read the code to memory and pass it to the interpreter via command line.

Errata: The database might be of any kind, but must run as a service, free from the contraint of not being able to write.

Speaking of reading, grant read access only to what you need.

I remind you that we are using command line because we have to. Using the command line is not what makes it safe.


Addendum: Due to the comments, I have considered if it is possible to use option 3 with file write access, without going into what I describe below (using two operating system users). It is vialbe to use option 3, on a single operating system user, with file write access. As long as the operating system user has only right over a constrained area. That would also allow the scripts to communicate/interfere with each other (the same would apply for option 2).


On the other hand, if you need to grant write access to the scripts (or if you don't want to use the command line), you will have to split your server solution in two operating system users. Run code under the first user to take requests, managing scripts, calling code to run as the second user and responding. Run code under the second user to run the scripts.

Have the code of the first user place the script in a temporary folder, grant write access on the folder to the second user. Make sure that folder is the current path for the script you run.

The script will be able to use the file system inside that folder, and you can use the first user to wipe it.

There is still a problem: they can try to fill the disk with a script. Even if you set a small quota for the second user, they may cause a denial of service. To mitigate this, you may have code running as the first user to monitor the file system (not pooling, register a notification from the OS, being asp.net that would be using FileSystemWatcher) and if the script is using too much space you can kill it, and flag it as dangerous.

You can continue to use option 3 with that setup. Yet, you do not need to. You can use option 2:

Store all scripts in Db and create temporary file when execution is required. Provide tmp file to interpreter and remove it once job is done.

If you do, you may or may not grant write access on the script themselves. It is possible that the script has code that reads or write itself as some sort of metaprogramming technique. You need to consider if you want to allow this.

You need metadata. Either a document oriented database or a relational database will work.


In fact, if you have the two operating system users solution, you can even use option 1:

Store all scripts in files and provide script files to interpreters.

Just do not grant the second user right to write the scripts. Preventing the scripts to mess with other scripts.

As I said at the start, in option 1 you can allows the scripts can communicate/interfere with each other. This is happens when one script write to a file, and another one reads. If you want to prevent this, either don't grant them write access or don't use option 1.

Addendum: There is a possible side effect of option 1 with write access: scripts could create other scripts. If this is a problem, it would be better to keep a list of the "valid" script somewhere else.


Other concerns

You have said (regarding option 1):

This also allows to change scripts externally using user's favorite text editor and allows to debug scripts.

That should not be a concern, because you are providing a web GUI, which the user should be using. If you need to provide a text editor, build it in the GUI. Debugging is probably out of the question.

(...) multiple users could be accessing to and executing same files at the same time, since they operate from web GUI

You can control that from your web code. You may – for example – queue requests to prevent too many script running at once (or the same one running multiple times, if that is really a concern).


You have said (regarding option 3):

Problem with this approach is that I need to heavily escape the script code in order to paste it as command line argument.

You could create an executable that reads the code (from where you have it stored) and outputs the code (with whatever preprocessing you need to do) to standard output. Then redirect the standard input of the interpreter to take the output of that executable. A.k.a Piping.

  • Thank you for you answer. I have a question about your statement regarding option 3. The scripts cannot use the file system. Is that so? I cannot read other files from FS with this approach? Also I am trying to figure out if it is possible to run Python.exe process and feed source code to it's standard input using Process.StandardInput.Write(...) and then provide input data using some other stream. – Roman Artemov Jun 8 '17 at 6:58
  • 1
    @RomanArtemov that is not a constraint of the option 3. That is a contraint of running your whole solution without write acess to the file system... which forces you to use option 3. Splitting your code across two operating system users allows you lift that constraint (then you can use option 3 with file system acess, or any option really). - About Python, I'm not really familiar with the interpreter, if StandardInput.Write works correctly depends on how it handles input. Experiment. – Theraot Jun 8 '17 at 7:06
  • @RomanArtemov using a single operating sytem user, with file write access, under option 3, should be viable. I have thusly updated the answer to mention that. I also added a few errata, and an addendum about option 1. – Theraot Jun 8 '17 at 7:29
  • I think that only read access will be sufficient for scripts. Also running them under low access user does not seem to be a problem, since ProcessInfo class allows to specify domain user and pass under which process should be started. – Roman Artemov Jun 8 '17 at 7:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.