6

Assuming that a VM runs a JIT compiler on otherwise "interpreted" code, such as a line by line interpreter or some form of bytecode/IL code and determines that it can create optimised native code for some part of what is running, what mechanism is actually used to "emit" and then call the emitted code?

If I was writing a normal compiler, I couldsimply write the desired binary to a file, to be run later by the system in whatever manner it usually would be, but how would a JIT compiler do that during runtime, running the new code within the same process that created it? Would it simply write the raw bytes into an array of contiguous memory, cast that memory/arrays pointer as a function pointer, and call it? Or are there some other functions available in low level languages to facilitate this?

5
  • 3
    stackoverflow.com/a/27627308/8458043 this gives a good and simple explanation. Or do you need more details?
    – Darem
    Aug 25, 2022 at 7:27
  • That looks pretty much like what I was looking for, I'm not sure how I didn't find it in my googling before I asked this question! And it looks like it was my initial thought of 'write binary to array/buffer, and then jump to that location' which I thought seemed a little too simple to be the true answer. But im now down the rabbit hole, so we shall see
    – AIWalker
    Aug 25, 2022 at 7:32
  • It is a little bit more complicated. Because you have to give the buffer the permission to execute code. This answer explains it a little bit more in depth stackoverflow.com/a/18477070/8458043. But basically your idea was right.
    – Darem
    Aug 25, 2022 at 7:34
  • Yes i saw that execute permission being a problem while delving into the other questions linked and saw that as well, and i also found that same answer. Still not sure why I never found this while searching earlier, but thank you very much for pointing me to these questions & answers :)
    – AIWalker
    Aug 25, 2022 at 7:38
  • 1
    No problem glad I could help :)
    – Darem
    Aug 25, 2022 at 7:39

3 Answers 3

9

If I was writing a normal compiler, I couldsimply write the desired binary to a file, to be run later by the system in whatever manner it usually would be, but how would a JIT compiler do that during runtime, running the new code within the same process that created it?

There is no reason a JIT compiler couldn't do the same thing as well. In fact, that is exactly what the MJIT compiler in the YARV Ruby VM does: it generates C code, writes it to a file, calls GCC to compile that file into a library, then links the library into the current process.

And it turns out this is actually a lot more efficient than it sounds!

Or are there some other functions available in low level languages to facilitate this?

This is not really a feature of the language, more of the host platform. Most modern platforms have a concept which allows dynamically linking code into a process, in Windows, those are called "DLL", in macOS "dyld", in Java "JAR" or "class file", and so on.

Would it simply write the raw bytes into an array of contiguous memory, cast that memory/arrays pointer as a function pointer, and call it?

That is another possibility.

A third possibility is that most modern languages have a way of calling "external" code (often called a "Foreign Function Interface" or "FFI"). So, you could just use the standard FFI API to call that generated code.

A fourth possibility is that many language implementations use native code somewhere, so they need to have a way of interacting with that code anyway. For example, in the Oracle HotSpot JVM, the low-level functionality is implemented in C++ and called as native code. (For example, the code that implements the iadd bytecode is written in C++, not in Java, and is executed natively.) So, the language implementation must already have a way of calling native code anyway.

1
  • in regards to YARV writing a file, calling and linking, aswell as DLLs etc, I did briefly consider that but kinda disregarded it as that seems like a lot of extra and potentially slow work, especially writing a file which is a lot slower than working in memory. If you write extra files at runtime like that don't you run the risk of a lot of junk being left behind in the event of a crash? Or does the VM just clean up next run? an existing DLL being used/called and linked later makes sense for one that already exists, but it didn't seem like it would be efficient or fast enough for JIT
    – AIWalker
    Aug 25, 2022 at 19:29
2

The reality on a modern operating system is that you can't just run any compiled code, you need the operating system to give you permission for that. For example, on MacOS getting that permission will be difficult, on iOS it can be impossible.

You will carefully check what functionality for this is available on your operating system. It may be that you can just generate a file just like a linker would, mark it to be "executable" and load it just like a dynamic library. If you are lucky, you can do that with data in memory: You tell the operating system "I want this data to become executable code" and the OS does it for you.

On some OS you may need permission from the manufacturer to do this. For example, an iPhone is supposed to be secure. It wouldn't be very secure if anyone could run code that has never been reviewed, so you would need permission from Apple to be able to do this. At the extreme, in iOS 16 users will be able to turn on a security setting that makes JIT compilation impossible. If you have some nation state seriously after you, that's the setting that you would use, because it prevents many possible attacks. Also if you are just paranoid and think they are after you.

2
  • If you're relying on lack of JIT as a security boundary, you're on shaky grounds to begin with. You could do anything without a JIT by simply interpreting the code. It would just perform worse. So a JIT as a concept doesn't really introduce any security risks, although like any piece of software, JITs can have security holes. A JIT is just an optimisation technique, not some back door into the underworld. The only place where it would be one is in an operating system that has no concept of privilege limitations. Aug 26, 2022 at 11:16
  • @AllonGuralnek Apple's App Store chooses to rely on JIT as a security boundary. Jan 19, 2023 at 12:19
0

You simply allocate some memory, put the code in the memory, and call the code. This is all that the OS's program loader really does, anyway. It puts the code into memory, and then it calls a certain memory address, which is the starting point of the program.

There are a few unique complications, but not many:

  1. You may need to flush (reset) the instruction cache so the CPU reads the instructions you just wrote, instead of whatever was there previously. Self-modifying code is rare, so many CPUs don't detect this automatically.
  2. You may need to mark the allocated memory as executable. Modern CPUs will not execute code unless the memory region is specifically marked as executable. This is an extra layer of security to try and catch bugs where you accidentally call data (like a web page) which could lead to evil people installing viruses. If you're calling some memory on purpose, and it's not a bug, you need to tell the CPU that.

Would it simply write the raw bytes into an array of contiguous memory, cast that memory/arrays pointer as a function pointer, and call it?

In C or C++, this is a typical way to do it. It's undefined behaviour according to C and C++, but it's defined behaviour for the CPU, as long as you are sure that the compiler does what you expect, since compilers are allowed to wrongly compile undefined behaviour. In practice, compilers tend to compile this the way you expect, even with optimization turned on. If you want to guarantee it, you might use inline assembly code to do the call.

If you are using a language more like Java, Python, or Ruby, you might have considerably more difficulty trying to convince the interpreter to call some random byte buffer.

If you wrote your compiler in assembly language (!) then of course you would just write a call instruction.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.