Does exist a reflexive compiled language? [closed]

Question

I just wonder if exists a compiled language that can modify it's own machine code.

I know that in most common operative systems the executable code is protected during execution time, but maybe if you're using some specific a permissive operative system, you can use a language that allow you use reflexive features over it's own code.

I found that some assembly programmers used reflexivity so maybe someone invented some high level language that allows you to compile reflexive machine code.

UPDATE The real case We are building a high performance scientific program. We are using in C as a portable assembly. We usually check the generated assembly code to improve our code making it faster. Looking inside we figured that a good way to improve the code is that C have some facilities of machine code reflexivity.

Example: Imagine this code:

void func (int b) {
int i;
for ( i = 0 ; i < 10 ; i ++) {
  if ( i + b == 2 ) do_some();
  do_other();
}

Would be more performant if you could make a reflexive approach

void util(int i, int b) {
  if ( i + b == 2 ) {
    do_some();
    delete_util(); //this will delete the util function and just continue where is called.
  }
}

void func(int b) {
int i;
for ( i = 0 ; i < 10 ; i ++) {
  util(i, b);
  do_other();
}

I know there are a lot of reflexive languages that are interpreted or compiles to VMs, but because of that are less performant, then not what I need.

Answer 1

First, the very definition of compiled languages is suspicious (actually, it is language implementations that are mostly compiled - and some of them compile to bytecode, but even in most C implementatons the printf format string can be viewed as interpreted by the C standard library; so there is a continuous spectrum between interpreters and compilers).

^{(In the rest of my answer, I am implicitly focusing on Linux systems, commonly used for HPC)}

Then look for homoiconic languages, notably Common Lisp and Io. Notice that the SBCL implementation of Common Lisp is compiling to machine code at every REPL interaction. Also, SBCL can be configured to generate code as efficient as optimized C.

BTW, usually code is not self-modifying (because on current tablet/laptop/desktop/server processors it makes the CPU caches very unhappy, and because reasoning about self modification is tricky), but several applications (and language implementations) are generating fresh machine code at runtime, then running it -in the same process. You can use JIT compiling techniques (libjit or GCCJIT etc...), or simply generate C (or even C++) code at runtime, compile it into some shared object, and dynamically load that fresh code e.g. with POSIX dlopen (on current systems, compilers are quick enough to make that compatible with human interaction; see my GCC MELT as an example.).

Notice that cleverly emitting some C code at runtime (you'll better at least represent the full AST of the emitted code in memory before writing it to some generated /tmp/emittedcode.c file), then compiling it with gcc -O3 -Wall -fPIC -shared /tmp/emittedcode.c -o /tmp/emittedcode.so, then dlopen-ing the freshly produced /tmp/emittedcode.so plugin, can give you very efficient runtime generated code, as efficient as your original program, if you emit the C code with care. There is no need to delete code, just use function pointers obtained by dlsym-ing your generated plugin (you could indirectly have closures, or C++11 std::functions, or callbacks above dlsym-ed function pointers). In practice (see my manydl.c example) you can afford many hundreds of thousands of dlopen-ed plugins without doing any dlclose (because you'll mostly "leak" code memory, which does not grow that fast).

So with most metaprogramming techniques, a small piece of code is rarely (or never) self-modified, but the entire software system is modifying itself (often growing itself) by generating and using new code (hence, self-modification happens at the program level, not inside a small function); even when you program in assembler (which you should not do much today, because compilers are generating better code than human programmers realistically can) you rarely use self-overwriting code (in particular, because it is not efficient today).

Notice that GCCJIT being GCC-based, and LLVM too being part of a compiler suite, can both be configured and used to emit at runtime very efficient machine code. Of course the JIT-compilation, that is machine code emission, time will need to be bigger (to run the same optimizations as your favorite compiler), but you probably don't care. On the other extreme GNU lightning and tinycc are generating very quickly slow machine code.

Read also about multi-stage programming & dynamic software updating. Look into J.Pitrat's blog (he passed away in October 2019 and was the first French AI researcher) & read his books. See also the self-generated C code of his CAIA system, and look into the ongoing RefPerSys free software project.

If you are not familiar with compiler technology (and emitting C code is a form of compilation), you should take time to study it. Assuming you know some dialect of Lisp I recommend the Lisp In Small Pieces book by Queinnec (because most of compilation is not parsing, see this & that).

^{(If you care about performance, you really should benchmark. What is mattering is cache misses and pipeline stalling. Arithmetic and predictable jumps are nearly free, but an L3 cache miss (to your RAM modules) takes more than 100 nanoseconds, i.e. more time than several hundreds machine instructions; see this page and its answers section...; current processors are spending half of their time waiting for memory (L2 or L3 cache, RAM modules))}

Answer 2

First of all, while most modern operating systems prevent writing to code by default, all common systems provide a way of overriding this behaviour. This is necessary if you want to use a language or environment that has a JIT compiler (e.g. Java, .Net, node.js, etc.). So you don't need a specific os for this.

Secondly, many existing languages provide facilities for this, although the facilities vary in ease of use radically. For example, both the .net framework and Java virtual machines support libraries that effectively allow you to produce new classes that contain code generated using the assembly language representation of their virtual machine Instruction sets. Other languages, e.g. Javascript and Ruby provide a mechanism usually called "eval" that will take a string of text in the language itself and execute it. This is commonly thought of as a facility primarily of interpreted languages, but many languages originally intended to be interpreted can now be compiled to native code, so the distinction is less than you might think.

Lisp (and derivatives like Scheme and Clojure) takes a third approach: code has the same structure as data in these languages, and can be directly manipulated. This similarity of structure is called Homoiconicity and is a very useful property of languages for programs that manipulate code directly. Again, many such languages are compiled to native code prior to execution.