97

For most of my programming career, I've used the "build/compile/run" command in whatever IDE I'm working with to produce a runnable program. This is one button, pretty easy. As I learn more about different languages and frameworks, though, I see more and more talk of "build scripts" (ANT, Maven, Gradle, etc.) to get a project running. My understanding of these is that they are instructions to the compiler/linker/magical-program-maker that specify configuration details - minutiae.

I remember writing makefiles back in school, but I didn't see any special advantage then (we only used them when writing in a Unix terminal, where an IDE with it's handy "build" button wasn't present). Beyond that, I've seen other questions on here that discuss how build scripts can do more than just create your program - they can run unit tests as well as secure resources regardless of the host machine.

I can't shake the feeling that build scripts are important to understand as a developer, but I'd like a meaningful explanation; why should I use/write build scripts?

Responsibilities of Build Script and Build Server discusses the role it plays within a broader scope. I'm looking for specific advantages offered by a build script versus an IDE's "build/run" command or similarly simple methods.

  • 18
    the answers here have covered everything pretty well, but i wanted to mention the fact that when you click the "run" button in your IDE, it (almost invariably) executes a build script that your IDE generated. writing your own build script just give you more control over the process. – Woodrow Barlow Jun 29 '15 at 18:49
  • 6
    writing and sharing your own build script also means that anyone can build it with a process identical to what your process is even if the other person uses a different IDE. this helps with consistency. – Woodrow Barlow Jun 29 '15 at 18:51
  • 7
  • 1
    Build scripts are often important to understand as a developer, though certainly not always. Even in environments where build scripts are practical requirements, many "developers" won't care about them in the slightest. But the scripts are then important to 'builders' rather than to the developers. The last places I worked, most developers had practically zero connection to build scripts. – user2338816 Jun 30 '15 at 7:43
  • 3
    not everybody using IDE with "build" button – Vladimir Starkov Jul 1 '15 at 10:16
112

Automation.

When you are developing, only in the most simple projects will the default "build" button do everything you need it to do; you may need to create WS out of APIs, generate docs, link with external resources, deploy the changes to a server, etc. Some IDEs allow you to customize the build process by adding extra steps or builders, but that only means that you are generating your build script through the IDE commodities.

But developing a system is not only writing code. There are multiple steps involved. An IDE independent script can be executed automatically, meaning that:

  • When you commit a change to version control, a new build can be launched automatically by the server. It will ensure that you have not forgot to commit anything needed for the build.

  • Similarly, after the build is done, tests can automatically be run to see if you broke something.

  • Now the rest of the organization (QA, sysadmins) has a built product that

    a) is perfectly reproducible just from the control version.

    b) is common to all of them.

Even when I have been working as a one man team I have used scripts for that purpose; when I had developed the fix I would commit to SVN, export the SVN back in another directory and use the build script to generate the solution, which would go then to Preproduction systems and later to Production. If a few weeks later (with my local codebase already changed) someone complained of a bug, I would knew exactly which SVN revision I would have to checkout in order to properly debug the system.

  • 4
    This is the best answer IMHO. Everyone is right with their idea, but this one really shows why you want build scripts. Because they support automation which is going forward as you project expands will save you tons of time. – Alexus Jun 29 '15 at 17:00
  • 16
    @Alexus Not just time, but dumb mistakes, too. ;) "Repetition leads to boredom. Boredom leads to horrifying mistakes. Horrifying mistakes leads to, 'I wish I was still bored.' " (You could measure dumb mistakes in time, too, but it's important to realize that it's saving you time from several causes.) – jpmc26 Jun 29 '15 at 20:27
  • @jpmc26 being there - done that :D – Alexus Jun 29 '15 at 20:30
  • 2
    Even for one-man projects, running a local Jenkins server for automating the "build in separate directory" can help. I'm working on something under Windows that I need to confirm also cross-compiles, so having Jenkins build and run tests, then build the cross-compiled version makes me much more confident (and efficient, of course!) when I need to release bug fixes. – Ken Y-N Jun 29 '15 at 23:13
  • 4
    @JonasGröger Automation takes out one of the biggest variables: humans. – CurtisHx Jul 1 '15 at 11:33
34

Like code, a build script is executed by the computer. Computers are exceptionally good at following a set of instructions. In fact, (outside of self-modifying code), computers will execute the same sequence of instructions exactly the same way, given the same input. This delivers a level of consistency that, well, only a computer can match.

By contrast, us water filled fleshbags are just downright despicable when it comes to following steps. That pesky analytical brain has a tendency to question everything it comes across. "Oh...I don't need to that", or "Do I really to use this flag? Eh...I'll just ignore it." In addition, we have a tendency to get complacent. Once we've done something a few times, we start believing that we know the instructions, and don't have to look at the instruction sheet.

From "The Pragmatic Programmer":

In addition, we want to ensure consistency and repeatability on the project. Manual procedures leave consistency up to change; repeatability isn't guaranteed, especially if aspects of the procedure are open to interpretation by different people.

In addition, we are HILARIOUSLY sluggish in executing instructions (compared to a computer). On a large project, with hundreds of files and configurations, it would take years to manually execute all the steps in a build process.

I'll give you a real world example. I was working some embedded software, where most of the code was shared across a few different hardware platforms. Each platform had different hardware, but most of the software was the same. But there were little pieces that were specific to each hardware. Ideally, the common pieces would be placed in a library and linked into each version. However, the common pieces couldn't be compiled into a shared library. It had to be compiled with every different configuration.

At first, I manually compiled each configuration. It only took a few seconds to switch between configurations, and wasn't that big of a pain. Towards the end of the project, a critical defect was discovered in the shared portion of code, where a device would essentially take over the communication bus. This was BAD! Really bad. I found the bug in the code, fixed it, and recompiled every version. Except one. During the build process, I got distracted, and forgot one. The binaries got released, the machine was built, and a day later I get a phone call saying the machine stopped responding. I check it out, and discovered a device had locked the bus. "But I fixed that bug!".

I may have fixed it, but it never found its way onto that one board. Why? Because I didn't have an automated build process that built every single version with 1 click.

  • 2
    And you can go for coffee while the build script is runnning! – Peter Mortensen Jul 1 '15 at 19:48
15

If all you ever want to do is <compiler> **/*.<extension>, build scripts serve little purpose (though one can argue that if you see a Makefile in the project you know you can build it with make). The thing is - non-trivial projects usually require more than that - at the very least, you'll usually need to add libraries and (as the project matures) configure build parameters.

IDEs are usually at least that configurable - but now the build process relies on IDE-specific options. If you are using Eclipse, Alice prefers NetBeans, and Bob wants to use IntelliJ IDEA, you can't share the configuration, and when one of you pushes a change to the source control, they need to either manually edit the IDE-created configuration files of the other developers, or notify the other developers so they'll do it themselves (which means there will be commits where the IDE configuration is wrong for some of the IDEs...).

You also need to figure out how to do that change in each and every one of the IDEs used by the team, and if one of them doesn't support that particular setting...

Now, this problem is culture-dependent - your developers might find it acceptable to not have the choice of IDEs. But developers that have experience with an IDE are usually both happier and more efficient when using it, and text-editor users tend to get religiously zealot about their favorite tools, so this is one of the places where you want to give freedom to the developers - and build systems allow you to do just that. Some people might have a build system preference, but it's not nearly as fanatic as IDE/editor preferences...

Even if you get all the developers to use the same IDE - good luck convincing the build server to use it...

Now, that's for the simple build process customizations, that IDEs tend to provide nice GUI for. If you want more complex stuff - like pre-processing/auto-generating source files before the compilation - you'll usually have to write a pre-build script in some basic text area inside the IDE's configuration. Which build systems you would still have to code that part, but you can do it in the actual editor you write the code in, and more importantly: the build system's framework itself usually provide some support for organizing these scripts.

Finally - build systems are good for more than just building the project - you can program them to do other tasks that everyone in the team might need performed. In Ruby on Rails, for example, there are build system tasks for running database migrations, for cleaning the temporary files, etc. Putting these tasks in the build system ensures that everyone on the team can do them consistently.

  • 2
    It's not just a matter of different IDEs. Some developers hate and loathe IDEs of all types. – David Hammen Jun 29 '15 at 17:38
  • 2
    @DavidHammen I'm one of these developers(though I've configured my Vim so hard is's possible that I might have turned it into an IDE...), and while writing that paragraph I was automatically writing "editors" and had to fix it to IDE, because I think the fixture on IDEs is an important part of my answer. The asker clearly comes from the IDEs world, and the question talks about IDE-less development as something you are forced to do when there is no proper IDE available. The point of this answer is to show how build systems are beneficial even for teams composed purly of IDE users. – Idan Arye Jun 29 '15 at 22:01
  • I too am one of those developers. I don't want my fingers to have to leave the keyboard. Doing so interrupts my thought processes. – David Hammen Jun 29 '15 at 22:25
  • 1
    I disagree. I prefer IDEs. They allow me to not care about names, easy lookups, refactoring, nice UI. I mean, if an IDE can do something for me that I would otherwise do with sed ack etc, I use that. – ps95 Jun 29 '15 at 22:40
  • The point is that with build scripts every developer on the team can use whatever they prefer, while with the IDE's build functionality you are forcing everyone not only to use an IDE, but to use the very specific IDE the project is configured for(unless you have it configured for multiple IDEs, and good luck synching that...) – Idan Arye Jun 29 '15 at 23:16
13

Many IDEs simply package up the commands used to build something and then generate a script and call it!

For example, in Visual Studio, you can see the command-line parameters for a C++ compile in the 'command line' box. If you look closely at the build output you'll see the temporary file that contains the build script that was used to run the compile.

Nowadays, it's all MSBuild, but that is still run directly by the IDE.

So the reason you use the command line is that you are going straight to the source and skipping the middle man, a middleman who might have been updated or require a heap of dependencies that you just don't want or need on a headless server that acts as your continuous integration (CI) server.

In addition, your scripts do more than the usual developer-oriented steps they're designed for. For example, after a build, you may want your binaries packaged and copied to a special location, or an installer package created, or a documentation tool run on them. Many different tasks are performed by a CI server that are pointless on a developer machine, so whilst you could create a IDE project that performed all these steps you'd then have to maintain two of them - a project for developers and another for builds. Some build tasks (static analysis for example) can take a long time that you wouldn't want for developer projects.

In short though, it's just easier - create a script to do all the things you want and it's quick and simple to kick that off on the command line or a build server configuration.

9
make

is a lot easier to remember and type than

gcc -o myapp -I/include/this/dir -I/include/here/as/well -I/dont/forget/this/one src/myapp.c src/myapp.h src/things/*.c src/things/*.h

And projects can have very complex compilation commands. A build script also has the ability to only recompile the things that changed. If you want to do a clean build,

make clean

is easier and more reliable once set up properly than trying to remember each and every place an intermediate file may have been produced.

Of course, if you use an IDE, clicking a build or clean button easy, too. However, it's much more difficult to automate moving the cursor to a specific place on the screen, especially when that place may move if the window moves, than it is to automate a simple text command.

4

How else would you do it? The only other way is to specify one long command line command.

Another reason is that makefiles allow incremental compilation, which speeds up compile time a lot.

Makefiles can also make a build process cross-platform. CMake generates different build scripts based on the platform.

Edit:

With an IDE, you are tied down to a particular way of doing things. Many people use vim or emacs even though they do not have many IDE-like features. They do it because they want the power these editors provide. Build scripts are necessary for those not using an IDE.

Even for those using an IDE, you might want to really know what is going on, so the build script offers you the low down details of implementation that a GUI approach doesn't have.

IDE's themselves use often build scripts internally too; the run button is just another way of running the make command.

4

The above answers cover a lot of good ground, but one real-world example that I'd like to add (that I can't add as a comment due to no karma), is from Android programming.

I'm a professional Android/iOS/Windows Phone developer, and I use Google services APIs (mostly Google Maps) a lot.

In Android, these services require that I add a keystore, or a type of developer ID file that tells Google that I am who I say I am, to a developer console. If my app is compiled with a different keystore, the Google Maps section of the app will not work.

Instead of adding and managing a dozen keystores to the developer console, only one of which can actually be used to update the app anyway, I include this keystore in our secure repo and use Gradle to tell Android Studio exactly what keystore to use when building for "debug" or "release". Now, I only have to add two keystores to my Google developer console, one for "debug" and one for "release", and all of my team members can clone the repo and get right to developing without having to go in to the dev console and add the SHA hash of their particular keystore, or worse, making me manage them. This has the added benefit of giving each team member a copy of the signing keystore, meaning that if I am out of the office and an update is scheduled, a team member only needs to follow a very short list of instructions in order to push an update.

Build automation like this keeps builds consistant, and reduces technical debt by cutting way down on set-up time when we get a new developer, a new machine, or when we have to re-image a machine.

1

Build script advantages:

  • changes look like code (for example, in a git diff command), not like different checked options in a dialog

  • creating more output than a simple build

In some of my previous projects, I've used the build scripts to:

  • generate the project documentation (doxygen-based)
  • build
  • run unit tests
  • generate unit test coverage reports
  • pack binaries into a release archive
  • generate internal release notes (based on "git log" messages)
  • automated testing
0

Often you can call the "build" button in an automated way (Visual Studio accepts command line arguments, for example). People write build scripts as soon as they need something that the build button cannot provide.

For example, most IDEs will only let you build one platform at a time. Or only one language at a time. Then there's what you do with the built outputs: can your IDE roll them all up into an install package?

protected by gnat Jun 30 '15 at 22:40

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.