35

When designing an own programming language, when does it make sense to write a converter that takes the source code and converts it to C or C++ code so that I can use an existing compiler like gcc to end up with machine code? Are there projects that use this approach?

  • 6
  • 4
    If you look past C you'll see that C# and Java also both compile to intermediate languages. You're saved from having to redo a lot of work that someone else has already done by targeting an intermediate language instead of going straight to assembly. – Casey Oct 2 '14 at 16:00
  • 1
    @emodendroket However, C# and Java compile to an IL that is designed to be an IL in general and for C#/Java specifically, so in many ways CIL and JVM bytecode are more sensible and convenient as an IL than C could ever be. It's not about whether to use any intermediate language, it's about which intermediate language to use. – user7043 Oct 2 '14 at 16:08
  • 1
    Look at several free software implementations generating C code. And I hope you'll make your language implementation free software. – Basile Starynkevitch Oct 2 '14 at 17:21
  • 2
    Here is the updated link from @RobertHarvey's comment: yosefk.com/blog/c-as-an-intermediate-language.html. – Christian Dean Dec 29 '17 at 17:47
52

Tranlating to C code is a very well established habit. The original C with classes (and the early C++ implementations, then called Cfront) did that successfully. Several implementations of Lisp or Scheme are doing that, e.g. Chicken Scheme, Scheme48, Bigloo. Some people translated Prolog to C. And so did some versions of Mozart (and there have been attempts to compile Ocaml bytecode to C). J.Pitrat's artificial intelligence CAIA system is also bootstrapped and generates all its C code. Vala also translates to C, for GTK related code. Queinnec's book Lisp In Small Pieces have some chapter about translation to C.

One of the issues when translating to C is tail-recursive calls. The C standard does not guarantee that a C compiler is translating them properly (to a "jump with arguments", i.e. without eating call stack), even if in some cases, recent versions of GCC (or of Clang/LLVM) do that optimization.

Another issue is garbage collection. Several implementations just use the Boehm conservative garbage collector (which is C friendly...). If you wanted to garbage collect code (like several Lisp implementations do, e.g. SBCL) that might be a nightmare (you would like to dlclose on Posix).

Yet another issue is dealing with first-class continuations and call/cc. But clever tricks are possible (look inside Chicken Scheme). Accessing the call-stack could require a lot of tricks (but see GNU backtrace, etc....). Orthogonal persistence of continuations (i.e. of stacks or threads) would be difficult in C.

Exception handling is often a matter to emit clever calls to longjmp etc...

You may want to generate (in your emitted C code) appropriate #line directives. This is boring and takes a lot of work (you'll want that to e.g. produce more easily gdb-debuggable code).

My MELT lispy domain specific language (to customize or extend GCC) is translated to C (actually to poor C++ now). It has its own generational copying garbage collector. (You might be interested by Qish or Ravenbrook MPS). Actually, generational GC is easier in machine generated C code than in hand-written C code (because you'll tailor your C code generator for your write-barrier and GC machinery).

I don't know any language implementation translating to genuine C++ code, i.e. using some "compile-time garbage collection" technique to emit C++ code using a lot of STL templates and respecting the RAII idiom. (please tell if you know one).

What is funny today is that (on current Linux desktops) C compilers may be fast enough to implement an interactive top level read-eval-print-loop translated to C: you'll emit C code (a few hundred lines) at every user interaction, you'll fork a compilation of it into a shared object, which you would then dlopen. (MELT is doing that all ready, and it is usually fast enough). All this might take a few tenths of a second and be acceptable by end-users.

When possible, I would recommend translating to C, not to C++, in particular because C++ compilation is slow.

If you are implementing your language, you might also consider (instead of emitting C code) some JIT libraries like libjit, GNU lightning, asmjit, or even LLVM or GCCJIT. If you want to translate to C, you might sometimes use tinycc: it compiles very quickly the generated C code (even in memory) to slow machine code. But in general you want to take advantage of the optimizations done by a real C compiler like GCC

If you translate to C your language, be sure to build the entire AST of the generated C code in memory first (this also makes easier to generate first all the declarations, then all the definitions and function code). You would be able to do some optimizations/normalizations this way. Also, you could be interested in several GCC extensions (e.g. computed gotos). You'll probably want to avoid generating huge C functions - e.g. of a hundred thousands line of generated C - (you'll better split them into smaller pieces) since optimizing C compilers are very unhappy with very big C functions (in practice, and experimentally, gcc -O compilation time of large functions is proportional to the square of the function code size). So limit the size of your generated C functions to a few thousand lines each.

Notice that both Clang (thru LLVM) and GCC (thru libgccjit) C & C++ compilers offer some way to emit some internal representations suited for these compilers, but doing so might (or not) be harder than emitting C (or C++) code, and is specific to each compiler.

If designing a language to be translated to C, you probably want to have several tricks (or constructs) to generate a mixture of C with your language. My DSL2011 paper MELT: a Translated Domain Specific Language Embedded in the GCC Compiler should give you useful hints.

  • Are you referring to "Chicken Scheme?" – Robert Harvey Oct 2 '14 at 17:29
  • 1
    Yes.I gave the URL. – Basile Starynkevitch Oct 2 '14 at 17:30
  • Is it relatively practical to make a virtual machine, like Java or something, compile bytecode to C, then use gcc for JIT compilation? Or should they just go straight from bytecode to assembly? – Panzercrisis Oct 4 '14 at 15:42
  • 1
    @Panzercrisis Most JIT compilers require their machine code backends to support things like replacing a function and patching existing code with a jump/trap door. Aside from that, gcc specifically is... architecturally less suited to JIT compilation and other use cases. Check out libgccjit though: gcc.gnu.org/ml/gcc-patches/2013-10/msg00228.html and gcc.gnu.org/wiki/JIT – user7043 Oct 4 '14 at 16:49
  • 1
    Great orientation material. Thanks! – user183922 Feb 15 at 15:51
8

It makes sense when the time to generate full machine code outweighs the inconvenience of having an intermediate step of compiling your "IL" into machine code using a C compiler.

Typically domain-specific languages are written in this way, a very high level system is used to define or describe a process that is then compiled into an executable or dll. The time taken to produce working/good assembly is much greater than generating C, and C is quite close the assembly code for performance, so it makes good sense to generate C and re-use the skills of the C compiler writers. Note that it isn't just compiling, but optimising too - the guys who write gcc or llvm have spent a lot of time making optimised machine code, it'd be daft to try to reinvent all their hard work.

It might be more acceptbale to re-use LLVM's compiler backend which IIRC is language neutral, so you generate LLVM instructions instead of C code.

  • Seems like libraries are a pretty compelling reason to consider it too. – Casey Oct 2 '14 at 16:18
  • When you say "your 'IL'", what are you referring to? An Abstract Syntax Tree? – Robert Harvey Oct 2 '14 at 16:24
  • @RobertHarvey no, I mean C code. In the OPs case, this is an Intermediate Language halfway between his own high-level language and machine code. I put it in quotes to try and convey this idea that its not IL as used by many people (ie Microsoft's .NET IL for example) – gbjbaanb Oct 2 '14 at 16:32
2

Writing a compiler to produce machine code may not be much more difficult than writing one which produces C (in some cases it may be easier), but a compiler which produces machine code will only be able to produce runnable programs on the particular platform for which it was written; a compiler that produces C code, by contrast, may be able to produce program for any platform which uses a dialect of C which the generated code is designed to support. Note that in many cases it may be possible to write C code which is completely portable and which will behave as desired without using any behaviors not guaranteed by the C standard, but code which relies upon platform-guaranteed behaviors may be able to run much faster on platforms which make those guarantees than code which does not.

For example, suppose a language supports a feature to yield a UInt32 from four consecutive bytes of an arbitrarily-aligned UInt8[], interpreted in big-endian fashion. On some compilers, one could write the code as:

uint32_t dat = *(__packed uint32_t*)p;
return (dat >> 24) | (dat >> 8) | ((uint32_t)dat << 8) | ((uint32_t)dat << 24));

and have the compiler generate a word-load operation followed by a reverse-bytes-in-word instruction. Some compilers, however, would not support the __packed modifier and in its absence would generate code that wouldn't work.

Alternatively, one could write the code as:

return dat[3] | ((uint16_t)dat[2] << 8) | ((uint32_t)dat[1] << 16) | ((uint32_t)dat[0] << 24);

such a code should work on any platform, even those where CHAR_BITS isn't 8 (assuming that each octet of source data ended up in a distinct array element), but such code may likely not run nearly as fast as the would the non-portable version on platforms supporting the former.

Note that portability often requires that code be extremely liberal with typecasts and similar constructs. For example, code which wants to multiply two 32-bit unsigned integers and yield the lower 32 bits of the result must for portability be written as:

uint32_t result = 1u*x*y;

Without that 1u, a compiler on a system where INT_BITS ranged from 33 to 64 could legitimately do anything it wanted if the product of x and y was larger than 2,147,483,647, and some compilers are prone to take advantage of such opportunities.

1

You have some excellent answers above but given that, in a comment, you answered the question "Why do you want to create a programming language of your own in the first place? " with "It would be for learning purpose mainly," I'm going to answer from a different angle.

It makes sense to write a converter that takes the source code and converts it to C or C++ code, so that you can use an existing compiler like gcc to end up with machine code, if you are more interested in learning about lexical, syntax and semantic analysis than you are in learning about code generation and optimization!

Writing your own machine code generator is a pretty significant piece of work which you can avoid by compiling to C code, if it's not what you are primarily interested in!

If, however, you are into assembly program and fascinated by the challenges of optimizing code at the lowest level, then by all means, write a code generator yourself for the learning experience!

-7

It depends on what Operating System you are using if you are using Windows there is a Microsoft IL (Intermediate Language) Which converts your code into intermediate language so that it takes no time to get compiled into machine code. Or If you are using Linux there is a separate compiler for that

Coming back to your question is when you when designing your own language you should have a separate compiler or interpreter for that because machine does not know the high level language . Your code should be compiled into machine code to make it useful for machine

  • 2
    Your code should be compiled into machine code to make it useful for machine -- If your compiler produced c code as an output, you could put the c code into a c compiler to produce machine code, right? – Robert Harvey Oct 2 '14 at 15:56
  • yes. because machine doesn't the c language – Tayyab Gulsher Vohra Oct 2 '14 at 15:58
  • 2
    Right. So the question was "When does it make sense to emit c and use a c compiler, rather than emitting machine language or byte code directly?" – Robert Harvey Oct 2 '14 at 15:59
  • actually he asking to design his programming language in which he is asking that "converts it to C or C++ code" . So i am explaining this if your are designing your own programming language why you should use the c compiler or c++. if you are intelligent enough you should design your own – Tayyab Gulsher Vohra Oct 2 '14 at 16:02
  • 8
    I don't think you understand the question. See yosefk.com/blog/c-as-an-intermediate-language.html – Robert Harvey Oct 2 '14 at 16:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.