4

For a scientific simulation I need to write some computations in C++. Since this became extremely tedious, I built myself a small code generator: In a scripting language (Python) you put together a syntax tree consisting of assignments and mathematical operations. A self-written codegenerator transforms the tree into C++ code. (Good example for the visitor pattern, by the way.) That's all fine and ok.

Now the project is growing and demand for more sophisticated code comes. It's very difficult for me to see what tools could be useful. What do people typically do and use when they have to generate program code programmatically? (With Python as "generator language" or in general)

(Sorry, but in this case, Google is really not my friend. Google can not transform a task into search results, unfortunately.)

closed as off-topic by gnat, Ixrec, Bart van Ingen Schenau, Mason Wheeler, GlenH7 Mar 23 '16 at 15:07

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Questions asking us to find or recommend tools, libraries, programming languages, resources (including books, blogs, tutorials, and examples), or projects to undertake are off-topic here as they attract opinionated answers that won't have lasting value to others. You may be able to get help in The Whiteboard, our chat room." – Ixrec, Bart van Ingen Schenau, Mason Wheeler, GlenH7
If this question can be reworded to fit the rules in the help center, please edit the question.

  • 3
    I don't have an answer for you (this actually seems like a tool or poll question so I'm not convinced it has an answer) but I can at least say that I do the same thing you do. – Lightness Races in Orbit Mar 23 '16 at 10:42
  • 2
    Unfortunately this is indeed an opinion poll and/or a tool recommendation as currently written, so it'll get closed sooner or later. But it's worth mentioning that C++ template metaprogramming is arguably a form of code generation built into all C++ compilers, so that's another option to consider. – Ixrec Mar 23 '16 at 10:48
  • 1
    It's Programmers, so I should be allowed to ask question that are a bit more open than on StackOverflow. If not, where (the hell) can I ask this kind of question? I think it's finally an absolutely legitimate question. – Michael Mar 23 '16 at 10:54
  • 2
    @Michael: What you have been doing is creating a DSL (Domain Specific Language)-to-C++ translator/compiler. Perhaps that helps in your search. You might also want to look into compiler, parser and lexer technology. – Bart van Ingen Schenau Mar 23 '16 at 11:41
  • 2
    python.org/about/success/cog. It was the first match in a Google Search for "C++ Code generator from python" – Robert Harvey Mar 23 '16 at 15:48
3

So you want to translate some (yours) domain specific language (DSL) to some flavor of C++. I am doing exactly the same in my GCC MELT implementation. It is a Lisp-y domain specific language to customize the GCC compiler. See also this answer giving slightly more details, and this one giving relevant references.

Here are some advices; I cannot be more specific because I have absolutely no idea what your domain specific language is for. Is it Turing-complete (perhaps accidentally)? Probably yes!

  • if you never studied it, study compiler techniques (including lexing & parsing). They are highly relevant.

  • read Scott's book Programming Language Pragmatics (at least for inspiration).

  • consider (instead of developing your own DSL) embedding some existing interpreter, perhaps Guile or Lua. It might be considerably simpler.

  • be aware that designing and implementing a passable DSL which is compiled (perhaps to C++) is a lot of work (years!). Read the mythical man month, Hofstadter's law, etc... Perhaps you want (or not) to bootstrap your language implementation...

  • if your DSL is somehow useful (e.g. you are not the only one writing scripts in it), be aware that eventually some crazy user would code large scripts (many thousands lines) in it. So design the language seriously!

  • first, you need a well defined (in your head!) representation of the abstract syntax tree (which might not be a tree, but a graph) of the generated C++ code and you should build (in memory) the AST before emitting the corresponding C++ code

  • you might want to emit #line directives (referring to positions inside your DSL scripts). It is very useful (for debugging) but emitting them is quite hard.

  • you might need several other internal representations, between the DSL source and the generated C++ code, and your C++ code generator (actually a specialized compiler) is transforming some representations into another ones, and finally into an AST which is emitted as C++ code.

  • you should care about the memory model; so read about garbage collection techniques (at the very least, for terminology and concepts), see the GC handbook. You probably don't want a stupid user script to crash the computer or the process. So you need to handle memory (& memory leaks).

  • perhaps you might consider, instead of generating C++ code, to use JIT compiling techniques: GCCJIT, LLVM, libjit, asmjit, ...

  • perhaps SciLab, R, Octave, Julia might be relevant for your work (because they might avoid you to start your own DSL)

1

Do you need to generate C++, or compiled code? You can leverage LLVM to produce compiled output from scripted tooling. The clang intermediate language is verbose but powerful - its what C++ code (or any other supported language) gets parsed into before compilation. Alternatively you can generate C++ from tools that still leverage the Clang parser, so instead of manipulating the text, you manipulate the internal AST the parser holds.

For an example, look at cmonster, which is a (rudimentary) python wrapper for clang's C++ parser.

Not the answer you're looking for? Browse other questions tagged or ask your own question.