23

I've recently been using some build tools for a Nodejs project at work when I realized that most languages' main build tool/system use a different language than the underlying programming language itself.

For example, make does not use C or C++ to write scripts and ant (nor Maven) doesn't use Java as their language for scripting.

Newer languages like Ruby do use the same language for build tools like rake, which makes sense to me. But why hasn't this always been the case? What't the advantage to having a build tool that uses a different language from the underlying language?

  • 14
    Maybe the general-purpose language isn't a good enough fit for the specialist usage of the tool? For example, I wouldn't want to write make files in C! Sometimes a DSL is better. – Andres F. Sep 21 '15 at 18:16
  • 13
    Look at it from another angle: Why not use make to write application software? – whatsisname Sep 21 '15 at 18:31
  • 4
    This article is primarily about what the author believes makes Lisp great, but has some relevant information on why Ant uses XML instead of Java. – 8bittree Sep 21 '15 at 18:34
  • 6
    If you wanted to write a program in C that automates building C programs, wouldn't you just end up writing make again (which is implemented in C anyway)? – Brandin Sep 21 '15 at 18:59
  • 6
    @joshin4colours make is written in C. The language that make uses, however, is a declarative language that invokes the shell as needed. – user40980 Sep 21 '15 at 19:38
36

The choice of what programming language to use to get anything done must depend on the specific features of that goal and not on the other tasks related to that project.

A build tool does a very specific job, and no matter what language you're using for the main project, a build tool is a software by itself.

Trying to tie the main project and its build tool could be a very bad decision. What you should need with a build tool is mainly a fast process of simple rules, with a fast management of files and IO.

If languages like ruby do use themselves to implement their build tool, is related more on the features of that language than on the assumed need to keep everything written with the same language.

18

It's not impossible to write a build tool that uses a language such as C or Java as their scripting languages. However, build tools benefit from using more declarative scripting languages. Imagine the pain and boilerplate of writing a makefile (or build.xml, or whatever) in C.

Build tools benefit from the use of DSLs, a task for which C isn't a particularly good choice. C is too general for a build tool, and too concerned with error-prone & low-level details. For example, you do not want to be concerned with explicit memory-management in a build tool! So even if using C-like syntax, you'd probably be using garbage collection in your scripting tool, at which point why not simply use a dedicated DSL?

It makes sense that Ruby can be used for this, since it's a higher-level, more declarative language than C or Java. Also see: Gradle, sbt.

  • 13
    Imagine the pain and boilerplate of writing a makefile (or build.xml, or whatever) in C. Imagine the pain and mess of trying to express non-trivial conditional logic (you know, standard imperative stuff) in a declarative XML script. Oh, wait, you don't have to imagine; you've probably had to deal with it, and it was probably a mess and a half, wasn't it? Builds are one of the worst possible choices for a declarative scripting language, because serious problems quickly end up butting up against the limits of declarative-ness, and the ones that don't are simple enough to not need a build tool. – Mason Wheeler Sep 21 '15 at 18:57
  • 7
    @MasonWheeler There are cases where existing build tools have made me suffer, yes. None of them would have been helped by using a low-level C-like imperative language. I see something like Ruby as probably ideal: declarative and imperative enough, as needed. C would give me nightmares for this task. To me build systems are a good use case for (mostly) declarative DSLs: you care about what to do, not about how to do it (most of the time). – Andres F. Sep 21 '15 at 19:02
  • 2
    Fair enough. I've always figured C is one of those "if X is the answer you're asking the wrong question" technologies, to be perfectly honest; it's just that working with XML "scripts" has been a nightmare every time I've had to dive into one. I agree that a higher-level imperative language with DSL capabilities would be ideal. – Mason Wheeler Sep 21 '15 at 19:05
  • @AndresF.: Things like make files require a mixture of approaches. The desired output state is specified declaratively, but the actions necessary to achieve that state are specified imperatively. – supercat Sep 21 '15 at 20:39
  • 1
    @MasonWheeler I think that is as much a problem of the people who designed the config language than XML itself. The common mistake by authors of such languages is to assume that just because something is in XML, it will automatically be human-readable. (I know I've made this mistake more times than I'm comfortable with. It's so tempting.) A well-designed and lean configuration language can work just as well over XML as in any other form. – biziclop Sep 22 '15 at 10:46
12

Let’s give you a real world example.

About 15 years ago I worked on porting a large system written in C from Unix to Windows, it was about 3 million lines of code. To give you some idea of scale, it took over 24hr to compile on some of our unix systems (RS6000), windows could compile the system in about 4hrs.

(We also had 2 million line of code in our own interpreted language, but decided not to use our language for the build systems as it was never designed for file processing. Also we needed a build system to compile the C code that implemented our language.)

At the time the build system was written in a mix of shell scripts and make files, these were not portable to windows – therefore we decided to write our own build system.

We could have used C, however we decided to use python, there was few reasons. (We also rewrote our source code control system in python at the same time, this is was very intergraded with the build system, so object files for checked in modules could be shared by developers.)

  • Most of our code could be built with a few simple rules (only a few thousand lines of python for all platforms, Windows, VMS, and 6 versions of Unix) that were drived from the naming conventions of the files.

  • At the time RegEx was not very standard between C systems on different platforms, Python had built in RegEx.

  • A few modules needed custom build steps, Python allowed class files to be loaded dynamically. We allowed a custom class to be used to build a module (lib), based on having the python file with a magic name in the folder. This was the killer reason to use python.

  • We considered Java, but it was not shipping on all the platforms at that point.

(The UI for our source code control system used a web browser as that was portable across all the platform. This was 6 months before we had a internet connection. We had to download the browser over X25!)

  • Having worked on several bigish systems I sometimes feel like I have a handle on how development scales. Then I read stories like this. Sheesh. – dmckee Sep 21 '15 at 22:22
  • @immibis This was the fashionable thing to do 15 years ago. Even more so, this practice was actually endorsed and encouraged by the leading Unix vendors (SGI and DEC in particular, but IBM too). – oakad Sep 22 '15 at 5:56
  • 1
    People tend to forget that Unix used to mean "expensive". Windows won the desktop/workstation war by being cheaper. It's for the same reason that Linux later won the server war. – slebetman Sep 22 '15 at 8:33
  • 1
    If I want a computer to run software I have written myself, then I want it be flexible and have 101 options how the kernels is compiled etc. However if I am selling software to be installed on the customer’s computer, I want the customer to be running a OS that is not flexible, so that I know it will work on their computer if it works on mine. That was a big factor when looking at Linux as a software vendor - support just looked too expensive. Linux has won the hosted server software wars, where the server is provided by the software vendor – e.g. most web deployed software. – Ian Sep 22 '15 at 8:54
  • 3
    @slebetman It's only fair - Unix won the previous "war" by being cheaper (compared to the LISPM's and similar machines). It was absolutely horrid, awfully designed and crashing constantly (but it boots up so much faster than a LISPM! :P), using C as primary programming language (quite a high fall from LISP and similar), but it was much cheaper. In fact, many adopted it because it was free (there's been many free mutations long before Linux). In fact, I've met lots of old-school LISPers who preferred MS-DOS to unixes of the time. Yes, that horrible. – Luaan Sep 22 '15 at 13:00
3

The short answer to this question is that this is what a build tool is. A tool that automates the use of other tools, with the aim of building an end product from some source files. If we're writing the build script in the same language as the product itself, we're into the realms of language-specific tools, which would not be considered build tools in the same way.

This raises the question -- why don't compilers provide the sort of build automation that we're used to from tools like make? To which the answer is simply because make exists. The Unix philosophy of "do one thing, and do it well" suggests that it is not the responsibility of the compiler to provide this*. That said, I have personally been through my share of make headaches, and would be interested to try a compiler that provides it's own quote-unquote "native" build system.

For an example of a compiler designed this way, see Jon Blow's (as yet unreleased) Jai, a language intended as a C++ replacement. Links to talks and demos of the compiler are collected here. This language compiles to byte-code that runs within the compiler, with compilation to binary being a standard library provided function, accessible from the byte-code. The intention is to allow for both build automation and meta-programming within the language's native syntax.

* A look at Microsoft's Visual Studio will show you an example of the opposite philosophy. :)

2

The build tool is first and foremost a tool, just like 'gcc', 'ls', 'awk' or 'git'. You feed the tool an input (filename, parameters, etc) and it will do some stuff accordingly.

All of these tools allow you to pass in your input in ways that should be simple enough to write and understand. In the particular case of build tools, the input is a recipe file, a file that describes how your project goes from the source files to the finished product.

If your recipe is as simple as passing in the folder that contains the project and the compiler to use, then a declarative "language" is enough. Once the recipe becomes more and more complicated, with more logic, it becomes troublesome to handle it with declarative languages and more general purpose languages are used.

It should now be apparent that there is no connection between the language used to write your build recipes and the languages used to write your product code simply because they have different purposes and requirements. There is no connection even between the language the build tool is written in and the "language" the build recipes should be written in. Always use the best tool for the job!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.