15

I'm trying to add Continuous Integration to a project.

According to Wikipedia, one major piece of CI is automated builds. However, I'm confused about what, exactly, that means, as the CI and build automation articles seem to disagree.

Specific points of confusion: what does "automated build" mean in the context of:

  • a project using an interpreted language, such as Python or Perl?
  • building from source on an end-user's machine?
  • an application that has dependencies that cannot be simply pre-compiled and distributed, such as a database in an RDBMS local to the user's machine?
Improve this question
1
  • 2
    I've tagged with both builds and build because I didn't know which one to use.
    – user39685
    Dec 13, 2011 at 17:50

6 Answers 6

Reset to default
14

You are correct in noting that, for some technologies, a compilation step is not necessary. However, I recommend you take a broader view when interpreting the term "build automation". Think about "build" as including the following two major components:

  • The process for transformation source artifacts (code, database schema, documentation, etc.) deployed to an end user.
  • The application of quality assurance measures during said transformation

Automation, then, simply refers to making any--if not all--of those operations automatic (that is, not requiring manual intervention). This may include quite a variety of steps, depending on your technology:

Transformation steps:

  • Compilation
  • Linking
  • Packaging
  • Deployment
  • Data migration
  • Backup
  • Notification

Quality assurance steps:

  • Compiler warnings / errors
  • Unit tests
  • Integration tests
  • System tests
  • Deployment authentication

These days, good CI tools will let you address all of these concerns. Initially, most shops are interested in automating compilation of their code, since that's the first--and most visible--source of problems in conventional software development.

Improve this answer
21

An automated build is a description of a process that should cover the following basics:

  1. Fetch the latest code from Source Control
  2. Compile the latest code into the executable
  3. Run tests (unit tests, system tests, integration tests) against compiled code
  4. Deploy completed executable to a known location for deployment.
  5. Publish the results of the build.
    5.1 Successful Compilation, Unit test success

It is a hands-off, process that should run with zero manual intervention.

Improve this answer
2
  • 3
    Since it was explicitly asked, you might mention that steps that don't apply are optional. For example, if your app is a bunch of python scripts, step 2 might be nothing, or it might be something as simple as zipping the code into a single file. It's perfectly acceptable not to have a compile step.
    – Bryan Oakley
    Dec 13, 2011 at 20:05
  • @BryanOakley That's fair. The equivalent of not having a compile for scripts may be ensuring that all of your dependencies, if you have any, are properly gathered at this point. For example, any additional third-party libraries required by your Python script should be included so that they are included in all the following steps. This is unnecessary too, I suppose, if it is known that the target and build machine always have all the required libraries.
    – Sheldon Warkentin
    Dec 14, 2011 at 17:10
2

To my mind, an automated build is something that

  • happens automatically, either on a schedule or with each commit to source control
  • creates a set of artefacts that can be deployed simply to any server

The aim is to have a deployment process that can be repeated -- read: tested -- so that by the time you deploy to production, you have a fair degree of certainty that things won't go wrong. The less human interaction in the build and deploy processes, the safer your release will be.

If you have a non-compiled language, you can still build a site and zip it up to create a single artefact.

A good CI tool will allow you to script many tasks into the build process, including the running of unit tests. It will also keep records of your successful and unsuccessful builds, test coverage, etc. But none of that is part of what I'd define as an automated build. (ie. A good automated build process has these things, but a poor one doesn't fail to be called "automated build" because it lacks those things.)

I would suggest that integration/regression tests be run as part of the deployment process, rather than the build process (although, if you have a convenient environment, you can deploy with every build).

Improve this answer
2
  • It may also be useful to have a scheduled build and allow developers to kick off an automated build with one action (if it's two actions, it isn't really automated, is it?) In our case, the build for some systems is too long for kickoff each commit, so it's on schedule and on request.
    – David Thornley
    Dec 13, 2011 at 18:22
  • @DavidThornley: Yes. That's useful. Most CI tools allow you to kick off a build outside of your set schedule. But again, it doesn't cease to be an automated build because this option isn't there. It would cease to be an automated build if a developer always had to trigger it.
    – pdr
    Dec 13, 2011 at 18:31
1 
a project using an interpreted language, such as Python or Perl?

In the case of interpreted languages, things can be hit or miss. Some interpenetrated languages have compilers but more often than not there is likely not much of a need to use them. In that case, I would generally just scan the code for syntax and parsing errors in place of compilation or skip straight to running the tests on the code.

building from source on an end-user's machine?

To me this would mean that you can provide a single command that end-users can run to get the latest version of the program, compile it, configure and deploy as needed.

an application that has dependencies that cannot be simply pre-compiled and distributed, such as a database in an RDBMS local to the user's machine?

These would fall under the testing part of the continuous integration as you can automatically destroy and reconstruct the databases to ensure that the scripts are correct and that the program tests against it correctly.

Improve this answer
1

What you are discussing in your question are actually 3 different concepts:

Continuous Integration at it's core is making small changes and frequently synchronizing those changes with the "global truth". Instead of making a checkout and holding onto it for a week, a developer should work on tasks that can be completed within a day so that his code is never too far out of sync with the main repository.

In order to achieve this without causing his team pain (i.e. checking in source that doesn't build or breaks existing functionality). The developer has to verify that his code doesn't "break the build". If done manually, this adds additional overhead to the process of development (think of a project that takes a long time to build and/or has many inter-dependencies where a change to one line of code can impact the application in unexpected ways).

To mitigated this situation, we use other techniques to remove this overhead.

We use automated builds to checkout the source and build it optionally running automated tests that verify the application works as it should (this step is only as useful as the test suite).

A further step continuous delivery addresses your issue with database and other concerns. The idea here is to provide some level of versioning for the database and other factors of the environment so that we can confirm as quickly as possible that the application works in an environment as close to production as possible.

Improve this answer
1
  • 1
    Great points...it's too bad that most people don't read down to the bottom of the thread and vote.
    – hotshot309
    Oct 1, 2012 at 23:42
0

"Automated build" means that you can go from source control to a shippable package with one (schedulable) action (typically a shell script or batch file).

What exactly constitutes a build, in this context, depends a lot on what exactly it is you're shipping, how it gets delivered, and which steps are required for the various parts of your development stack, but in any case, you start with what's in source control, and you end up with a shippable product (or an error message and an angry project manager).

For a simple Python project, an automated build might consist of only two steps - checking out the sources, and copying relevant files to the correct directories. For more complex projects, it might involve things like:

  • compiling, linking
  • running automated tests
  • creating installer packages
  • installing
  • modifying database(s)
  • creating backups (in case you need to roll back)
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.