5

Is continuous integration about more than just automatically testing code, either periodically or after every commit to the main repository?

Before today, I'd heard the term CI, and about CI services such as Jenkins, and RunCodeRun, and assumed that it was merely about having a server independent of developers' machines that checked that the automated tests of a project passed, and alerting people if it doesn't.

However, Wikipedia's definition seems to be different.

If my understanding of what CI is is incorrect, is there any term that merely refers to automatically running tests on an independent server?

10

The gist of CI is to avoid any kind of long-term branches. If you have a team working on a product, a model opposed to CI is to pick a new requirement for a feature to develop for the product. Then the team creates a feature branch in the VCS, apart from the "trunk" or "master" branch, they develop and test the feature for several days in isolation, reintegrate it afterwards into the trunk and then do final integration tests (note that in between other parts of the team could have made changes to the trunk as well).

In CI, you split the new feature into several sub-parts, each of them creating still a working and compilable product, and integrate that sub-part into the main trunk very often, typically multiple times a day. There are no branches with big differences from the trunk, and if different groups of people are working on different features simultanously, they will daily ("continously") reintegrate all their changes to the code base. Almost all tests are always running on the "integrated" work, there will be almost no tests on non-integrated parts (except the tests each dev runs on his own local copy before committing his changes).

For this, you should have a server supported automatic build process in place and a lot of automatic tests which run at that server as well. This will give immediate feedback to the team if the changes they made might cause any problems on the integrated product. You won't need an "extra test phase" to check if the integration of a bigger feature into the product causes any problems. But those automatisms are just supporting tools to make CI work well, these steps are not the steps which actually define CI.

4

Automated testing would be a term for just running the tests every day, even if you only merged the code into a complete state once a month, or never.

Arguably a bad idea, because if you keep different strands of development in different long running branches you will be merging the tests and code at the same time. And if the features interact in any way, merging them should move some tests from passing to failing; if not, you weren't testing things adequately.

As a result, it becomes hard to figure out which tests failures are regressions, which are to be expected given the current state of integration, which have been temporarily disabled, and so on. In other words, interpreting the test results becomes a manual process, even if the execution of them is automated. Which risks wasting the effort put into automating the test in the first place.

One partial way round this is to have all automated test be strictly unit tests; literally one class at a time, mocking everything else with no exceptions. And then do all other testing either manually, or via some automated system outside the scope of the repository and its branches.

  • and what is the problem of failing tests that you expect to fail due to changes in the code? You should be welcoming these as it shows you what tests need updating as part of your ode builds (which is exactly why you run them in your CI process in the first place!) You could also be updating tests that you expect to be broken by your feature changes in the branches you apply the code changes anyway (ie where else would you fix your tests?!) – gbjbaanb Jun 22 '15 at 11:57
  • The problem with failing tests when you are not doing CI is that you can't fix them for weeks or months at a time. Which tends to mean the number of failing tests is rarely near zero, and so the information content of a failed test is small. – soru Jun 22 '15 at 16:17
  • "when you are not doing CI, you can't fix them for weeks or months at a time", sorry, but I do not buy that. When you integrate a feature branch into the main line (for example, at the end of the month), you just have to fix all failing tests immediately. Or in other words, you have to do proper integration (which includes all tests), but not "continuous integration": – Doc Brown Jun 27 '15 at 7:45
  • Not sure what you are disagreeing with - if you fix the integration tests for two interacting features when integrating, and integrate once a month, then that is a gap of 1-3 weeks between the test failure and the fix. – soru Jun 27 '15 at 8:47
0

Continuous Integration is a process of continually integrating new changes with other peoples changes. The goal being to eliminate merge problems developing over time when many developers are changing the code independently in different directions.

It is fashionable in commercial environments to have a centralised source control with an automated build server with various testing and reporting metrics run on the code in response to a code change whilst allowing developers to work in a seperate branch whilst integrating other peoples completed changes but not sharing their changes till their work is complete - and to call that continuous integration.

However, i disagree. This kind of approach will allow developers working on sperate features both making large radical changes to avoid integration until the first developer finishes their feature. Whenb they merging their changes into the common branch they then dump the kind of merge hell onto their collegue that continuous integration is intended to avoid.

True continuous integration has developers commiting often (many times daily) to the same branch, ensuring that all code changes are integrated before significant divergence can occur.

As an approach, CI provides some advantages in reducing the impact that unexpected surprises at merge time can have, but creates problems else where in the business process of releasing and delivering software. (e.g. trying to develop new features whilst fixing bugs for the imminant release)

My advice is to avoid religious devotion to a methodology, but to implement processes that support your team and business environment that reflect how you work, how your business works around you and enables you to focus on software delivery and not process management.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.