Amongst the successful and widely used JITs out there (e.g. JVM JITs, JavaScript JITs, LuaJIT), is there a strong consensus on how the JIT compiler should interface with an assembler? The plausible approaches seem to be an internal assembler (or more generally, any internal generation of machine code), using a library (e.g. libyasm), or using an executable assembler (e.g. gas, nasm, etc.). Obviously any of these approaches is possible and there's no way to sit down and formally prove that one of these approaches is a priori the "one true approach", but my question is whether real-life JITs have largely converged on one of these approaches.

Keep in mind, it's perfectly valid to answer "there is no widely accepted best approach and all of these approaches are used by major JITs" if that is indeed the case.

  • 1
    Just guessing, but since JIT compilers are often processing bytecode written for a VM of some sort, I think there's a good chance none of your plausible options are used and the reality is much more nebulous. Commented May 11, 2018 at 6:00
  • 5
    What makes you think that JIT compilers create assembly code in the first place? Why wouldn't they output the binary instructions directly? Commented May 11, 2018 at 8:02
  • 2
    JITs do not produce assembly language of any kind. They produce executable code, directly. Commented May 11, 2018 at 19:38
  • @BartvanIngenSchenau At some level there needs to be assembly-like intermediate data, even if it isn't human-readable assembly code. I was trying to include that possibility with "or more generally, any internal generation of machine code". Commented May 12, 2018 at 4:36

1 Answer 1


"there is no widely accepted best approach and all of these approaches are used by major JITs"

There is no widely accepted best approach and all of these approaches are used by major JITs. Plus some that you didn't list such as generating C source code.

For example, the brandnew MJIT compiler infrastructure released just days ago as part of the YARV Ruby execution engine, generates C source code which it then compiles using GCC. The author's reasoning for that is three-fold:

  1. Generating C and calling into GCC is actually much faster than you might think.
  2. The JIT automatically supports every platform YARV already supported without having to write a code generator for every platform.
  3. The JIT automatically supports lots of low-level optimizations, and every day the author doesn't spend on low-level optimizations, is a day he can spend on Ruby-specific high-level optimizations.

And there is a fourth, very exciting, reason, which is however still a long way in the future: a lot of code in the Ruby ecosystem is actually not written in Ruby but in C. This includes, but is certainly not limited to, the YARV VM itself, the YARV core libraries, parts of the YARV standard libraries, and lots of third-party libraries. Currently, those are usually delivered pre-compiled or are compiled upon installation. This means that to the Ruby execution engine, they are an opaque black hole, it cannot look inside. But, what if you deliver those in source code form, and compile them at runtime? Then, you can compile them together with the Ruby code that calls them (or is called by them), and perform optimizations across that barrier!

TruffleRuby and GraalPython are JIT-compiled without actually having a JIT compiler. So, how does that work? They use the Truffle AST interpretation framework. Truffle can specialize the interpreter while interpreting the AST. And what is an interpreter that is specialized to interpret only one program? Well, it is essentially a compiled version of that program! So, we end up with a compiled version of the program without actually ever having compiled it: the code is "JIT-compiled", but there is no real code generation, assembly or otherwise.

Note that Truffle, in turn, can use the Graal framework to generate native machine code, but it doesn't have to. (In which case it simply calls the Graal API, and doesn't care how the code gets generated, that is Graal's job.) It can also work in interpreted (and specialized) mode, or on the TruffleVM. The cool thing about the Truffle framework is that it is language-agnostic, and a program written in different languages can be represented in the same AST and interpreted, specialized and optimized together (including, for example, cross-language inlining), as long as there is a Truffle implementation for every language involved. Similarly to what I outlined above for the future of MJIT, there is a C interpreter based on Truffle, which allows TruffleRuby to interpret YARV C extensions together with the Ruby code and optimize them together. (Actually, the C interpreter was replaced with an LLVM bitcode interpreter, which makes it even more versatile.)

Language implementations built on top of the PyPy/RPython framework are also JIT-compiled without ever writing a JIT compiler. This works, because the RPython framework itself contains a JIT compiler, and unlike other implementations, where the JIT compiles the user program while it is running, this JIT compiler compiles the interpreter while it is running the user code! So, what you end up with is an interpreter compiled and optimized together with the program it is interpreting, specialized for that particular program, which is essentially equivalent to a compiled version of that program.

That way, every language implementation (e.g. Topaz) built on top of the RPython framework, gets a JIT compiler for free, and you only have to write an interpreter.

Eclipse OMR is a project to provide a framework of re-usable components for high-performance implementations of high-level languages. In fact, Eclipse OMR is essentially a refactoring of IBM J9, one of the fastest JVMs, into a set of language-agnostic, independently re-usable, orthogonal components. As a proof of Eclipse OMR's power, starting with Java 9, IBM J9 is actually built out of OMR components. Also, IBM is working on Ruby+OMR and Python+OMR. OMR includes among other things, a profiler, debugger, garbage collector, cross-platform threading library, cross-platform signal-handling, and a JIT compiler. This JIT compiler contains its own code generator.

Many JIT compilers use LLVM as their back end, so they don't generate assembly code but either LLVM bitcode, or they use LLVMs C++ API to create an in-memory instruction graph. LLVM then compiles that code using its own code generator.

Although GCC was not originally designed to be used as a library, there is libgcc, which can be used to either embed GCC into a larger application (such as using it for semantic assist in an IDE), or use it as a JIT compiler.

There are also libraries that are specifically designed to be used as compiler backends for JIT compilers, such as GNU libjit and GNU lightning.

So, in short, there is an infinite amount of possibilities used in the real world, including all of the ones you listed, and many more.

  • You forgot to mention libgccjit; it is a JIT library using GCC middle-end and back-end Commented May 12, 2018 at 11:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.