30

How would you define Continuous Integration and what specific components does a CI server contain?

I want to explain to someone in the marketing department what Continuous Integration is. They understand Source control - i.e. they use Subversion. But I'd like to explain to them properly what CI is. The Wikipedia Article never properly defines it, the Martin Fowler article only gives the following, which is basically a tautology followed by a vague explanation of 'integration':

Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible.

Update: I sent them this image, I couldn't find a simpler one.

enter image description here

Update 2: Feed back from the marketing chap (for the time when there was 3 questions):

I actually like all 3 answers – for different reasons. I feel like logging in just to thank them all!

Obviously he can't - so thanks on his behalf :)

Update 3: I've realised looking at the Wikipedia article that it does contain the principles which, when you take just the headings, is quite a good list:

  1. Maintain a code repository
  2. Automate the build
  3. Make the build self-testing
  4. Everyone commits to the baseline every day
  5. Every commit (to baseline) should be built
  6. Keep the build fast
  7. Test in a clone of the production environment
  8. Make it easy to get the latest deliverables
  9. Everyone can see the results of the latest build
  10. Automate deployment
2
  • 4
    o.O Your marketing department uses Subversion? Tempted to vote to close as "Too Localized"... ;-)
    – Jeroen
    Commented May 19, 2013 at 13:52
  • @Jeroen Yep, really, for files on the website. I did make them a nice big red button on a web page that says 'Do it' to update subversion on the server. :)
    – icc97
    Commented May 20, 2013 at 17:27

4 Answers 4

28

When someone changes the files that make up the software product and then attempts to check them in (in other words, attempts to integrate the changes into the main product code) you want to make sure that the software product can still be successfully built.

There is usually an external system, called the CI server, that either periodically or on every change, will grab the source files from version control, and attempt to build the product (compile/test/package). If the CI server can successfully do a build, the changes have been successfully integrated.

The CI server also has to be able to broadcast if the build failed or succeeded, so systems like Jenkins (one of the most used CI server today) will have ways to send emails/texts as well as a dashboard-like web interface with a bunch of information about current and past builds, who checked-in code, when things broke, etc. (On your image above, this would be the Feedback Mechanism.)

CI is important, because it ensures that on a continuous basis, you have a working product. This is important for all the developers who are working on the software product as well as for all the people who want to have access to daily releases of the software product, like QA.

7
  • 1
    Continuous Integration cares about build status, but also about tests. Commented May 17, 2013 at 13:24
  • 1
    and deployment, its not enough to compile, and run tests, but you also should ship the binaries to an environment so it can be tested by people (or automated tools) too.
    – gbjbaanb
    Commented May 17, 2013 at 13:36
  • 1
    Since the question asked for a simple explanation I left out many (most of the time project/team specific) details that might go into a CI system.
    – c_maker
    Commented May 17, 2013 at 14:01
  • Is this dependent on test-driven development? Code that compiles isn't always code that works right? Without tests to fail when the code isn't ready, how would the CI system know if the code really was successfully integrated?
    – mowwwalker
    Commented May 17, 2013 at 19:54
  • 1
    @user828584: In my answer, I imply that 'test' is part of a build. And as a side note, TDD is different than having tests to check quality. As a side-effect of TDD, you will have well written tests, but you can have tests without doing any TDD at all.
    – c_maker
    Commented May 17, 2013 at 20:12
35

I guess for your marketing department it is not important how CI works, but what CI means for new releases of your software.

CI will ideally mean that you can produce a new potentially releaseable version of your software every day, ready to be presented or sold to your customer, with some new features, functionality or bugfixes added. That does not mean you must deliver the new version every day, but you can if you want.

For example, if you have a new feature set planned to be officially released for the "2015"er version of your software, and you have parts of that feature set already coded and integrated today, the marketing guys can take the current version of your software and show it - more or less safely - at the next conference now in 2013. Without CI, they had to ask your team for an unplanned code freeze, every team member had to integrate the half-baked feature he is working on into the product, they might not have enough automatic tests ready, and guess what will happen at the conference - the "alpha version" of your 2015er release will have a much higher risk of crashing, especially when the new features are demonstrated.

1
  • 4
    +1 for approaching it from the perspective of the benefit it provides.
    – poke
    Commented May 17, 2013 at 17:34
18

You can't know what CI is unless you know what we used to do. Imagine a system with 3 parts. There's a UI that gathers data and puts it in the database. There's a reporting system that makes reports from the database. And there's some sort of server that monitors the database and sends email alerts if certain criteria are met.

Long ago this would be written as follows:

  1. Agree on the schema for the database and the requirements - this would take weeks because it had to be perfect as you'll soon see why
  2. Assign 3 devs, or 3 independent teams of devs, to the 3 pieces
  3. Each dev would work on their piece and test their piece using their own database copy, for weeks or months.

During this time the devs would not run each other's code, nor try to use a version of the database that had been created by someone else's code. The report writer would just hand add a bunch of sample data. The alert writer would hand add records that simulated report events. And the GUI writer would look at the database to see what the GUI had added. Over time, the devs would realize the spec was wrong in some way, such as not specifying an index or having too short a field length, and "fix" that in their version. They might tell the others, who might act on it, but usually these things would go on a list for later.

When all three parts were completely coded, and tested by their devs, and sometimes even tested by the users (showing them a report, a screen or an email alert) then would come the "integration" phase. This was often budgeted at several months but would still go over. That field length change by dev 1 would be discovered here, and would require devs 2 and 3 to make huge code changes and possibly UI changes too. That extra index would wreak its own havoc. And so on. If one of the devs was told by a user to add a field, and did, now would be the time the other two had to add it also.

This phase was brutally painful and pretty much impossible to predict. So people began to say "we have to integrate more often." "We have to work together from the beginning." "When one of us raises a change request [that's how we talked then] the others have to know about it." Some teams began to do integration tests earlier while continuing to work separately. And some teams began to use each other's code and output all the time, from the very beginning. And that became Continuous Integration.

You may think I am exaggerating that first story. I did some work for a company once where my contact chewed me out for checking in some code that suffered from the following flaws:

  • a screen that he wasn't working on had a button that didn't do anything yet
  • no user had signed off on the screen design (precise colours and fonts; the existence of the screen, its capabilities and what buttons it had were in the 300 page spec.)

It was his opinion that you don't put stuff into source control until it is DONE. He typically did one or two checkins a YEAR. We had a bit of a philosophy difference :-)

Also, if you find it hard to believe teams would be disconnected around a shared resource like a database, you really won't believe (but it's true) that the same approach was taken to code. You're going to write a function I can call? That's great, go ahead and do that, I will just hardcode what I need in the meantime. Months later I'll "integrate" my code so it calls your API and we'll discover it blows up if I pass null, I blow up if it returns null (and it does that a lot) it returns things that are too large for me, it can't handle leap years, and a thousand other things. Working independently and then having an integration phase was normal. Now it sounds like insanity.

5
  • 2
    We had a similar story in which we upgraded a custom application built on SP 2003 to SP 2007. Using VSS (yes, VSS :), each developer checked out a portion of their file, coded for a week and two, then when we integrated our code, boom, that when the problem started since our code deviated significantly. We fixed the integration problem for a month, and the project rented a hotel to accommodate those who live very far on weekdays. On that day, I learned to integrate code on a daily basis :-) Commented May 17, 2013 at 14:10
  • @OnesimusUnbound I remember getting bumped from VSS to Clearcase ... out of pan into fire. Years later after returning to the same company for drinks, I remember someone laughing at a fellow developer for mentioning this new source control called 'Git', "What do we need another source control system for??".
    – icc97
    Commented May 17, 2013 at 15:37
  • 1
    Thank you - i've strugged to understand CI before simply because I did not know what the alternative was
    – Rhys
    Commented May 17, 2013 at 16:31
  • Interesting. To me, this sounds more like comparing CI to waterfall. But you can use a non-waterfall methodology that avoids these problems (like iterative development) and not use CI. Commented Mar 18, 2015 at 21:06
  • @DanMoulding if I'm being agile and iterative and whatnot in my part of a larger thing that isn't integrated with what someone else is doing, I'm not doing CI. Heck, you can waterfall and CI. Design it all, code it all, test it all if you want - if everyone is using everyone else's code/schema/file layouts all the time, that's CI even if you're using waterfall. The connection is that without CI you live and die by BDUF because that's your only hope (and it turns out to be a faint hope) of having an integration phase of a reasonable length. Adopting CI allowed us to let go of BDUF. Commented Mar 18, 2015 at 21:12
0

I would define it this way but I might have misunderstood the concept altogether. Please correct me if so.

Continuous integration is an automated process that many software development teams find beneficial. Once setup it's fully automatic. This is how it works.

A developer's main job it to produce commits in various branches. Each commit can possibly introduce problems despite the author having tested it on their machine. Also sometimes branches drift far from the main release branch causing merge horror stories.

Enter CI. It locally merges each branch to the main one, compiles the whole software product and runs many/all software tests ever written about it. If anything fails it notifies the author of the breaking commit via email or chat message.

Sub-fields of CI are continuous build (verify that after each commit the software compiles), continuous testing(write computer programs which test the main computer program), continuous delivery(have a working version of the software product every day with all the new cool stuff added) and continuous deployment(automatically deploy daily if not to the main production server then at least to a shadow server(A/B testing)).

Another advantage of CI is allowing heterogeneous development environments. Each dev can work on their favourite OS and hardware, but the CI server is as close to the production environment as possible.

It's a very sensible tool for all environments.

Not the answer you're looking for? Browse other questions tagged or ask your own question.