1

So I decided to created a stack-based virtual machine and the result came out pretty good (in my opinion!). The first iteration was a basic toy VM.

I did a rewrite to make the VM work more realistically by giving it byte-oriented memory and data stack as well as n-byte oriented opcodes. After much testing, including a fibonacci in opcodes test, the result turned out beautiful, turing complete, and I really enjoyed making it.

I have now decided on another rewrite but to expand it into a scripting engine. I picked up a book called "Game scripting mastery" which talked about creating an entire scripting engine with custom scripting language from scratch.

It was a little overkill for what I wanted to do but the ideas of it were helpful but I'm hoping to take some better approaches than what the book shows.

As a scripting engine, my intention with this engine is to execute bytecode created from a normal C compiler by modifying the backend. My end goal is that the script engine can support every part of C.

In terms of the scripts, I've already implemented a header file format for each script and have each script sandboxed by allowing them to have their own memory and stack allocated when loading. The script's data layout is...

typedef struct __script {
    uchar       *pMemory, *pStack, *pInstrStream;
    uint64_t            ip, sp, bp;     // 24 bytes

    uint        uiMemSize;      // how much memory does script need?
    uint        uiStkSize;      // how large of a stack does script need?
    uint        ipstart;        // where does 'main' begin?
    uint        uiInstrCount;   // how many instructions does the code have?
    uint16_t    magic;          // verify bytecode
} Script_t;

In terms of all this, I only have two real questions.

Q1: Considering that each script is sandboxed, how can I have it so individual scripts can share data between one another?

I thought up two ways to solve this issue: 1 way is I could hack up a bytecode linker and link multiple compiled bytecode scripts into one big one and use them as static libraries

another method I thought up was that scripts compiled without main() can act as a DLL (dynamically loaded), then a script can load in a dllscript and then use a specific data whether function or variable similar to dlopen and dlsym used in linux.

Q2: What is the best way to bridge a public API between the VM and host application? (so that scripts can call native C code from host application side)

Since this VM is to run bytecode made from a C compiler, I'm pretty sure that each script will have to include a header file that contains the host-exported function prototypes for scripts to use though I'm not sure how that would work out.

Then there's the issue with exporting host-side variables like structs, unions, globals, etc. I haven't thought about this part because I'm still thinking about solving the issue of question #1.

What do you guys suggest?

  • Q1: are you referring to share data between different .c scrips intended to work together? – Adrian Maire Jul 20 '17 at 6:41
  • yessir, having different .c scripts compiled to bytecode working together at least in some, hopefully seamless form. – Nergal Jul 20 '17 at 6:43
0

Q1: Considering that each script is sandboxed, how can I have it so individual scripts can share data between one another?

A software made of several .c file is designed to work together, it make no sense to try executing a sub-set of them and they are designed to work in collaboration.

A sandbox has the reverse purpose: it intend to divide and protect individuals, by avoiding them having any impact on each others, avoiding communication, and avoiding collaboration.

Trying to make each .c file sandboxed, is IMO a wrong concept from the beginning and thus, any solution would be a patch, a security issue or both.

The solution is probably to consider this interaction closely from the sandbox perspective: A sandbox would encapsulate a software and not anymore a single .c file. We could imagine this sandbox to encapsulate several files, some memory management, some interface to devices, etc.. It would also require a way to define which files belong to the same software, maybe a meta-data file?


Q2: What is the best way to bridge a public API between the VM and host application? (so that scripts can call native C code from host application side)

Supposing you got your syntactical tree out of the object code, it seem feasible to detect external libraries: usually a table indicate expected external functions signatures. You may then call the specific C function in case of a known signature (e.i. standard library calls).

  • about the first question. several .c files working together are usually linked together though. Are you saying I should go to the trouble of making a linker for each bytecode file generated from each individual .c file? – Nergal Jul 20 '17 at 11:30
  • here's the github to the virtual machine github.com/assyrianic/Crown-Virtual-Machine – Nergal Jul 20 '17 at 11:41
  • That is the assumption I did (by asking you in a comment) . If that is not the case, what exactly are you interpreting? it is also what you seem to do from following sentence: " ...engine is to execute bytecode created from a normal C compiler..." – Adrian Maire Jul 20 '17 at 11:42
  • basically this is VM with enough features to run C code that's compiled to bytecode of course. Thus would I require building a bytecode linker or something else? – Nergal Jul 20 '17 at 11:43
  • So what is the Q1 about? and why C?: If you are doing a VM and not a scripting engine. For executing any executable format, what sharing of data do you need? – Adrian Maire Jul 20 '17 at 11:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.