Git Flow: should develop branch be unit tested in CI?

Question

We have a Git Flow type workflow (in rails) where our 'testing' (aka develop) branch has feature branches merged into it for UAT.

So:

• master has the current running production system.
• feature branches have a single feature (against a pull request)
• testing has a number of features that are in the development cycle

Currently we have CI (in travis) turned on for all feature branches, testing and master.

Should we run the (slow) unit test suite on testing, given that all code in it will have been tested at feature level (as master + feature branch)? The only things I can think of that could lead to testing failing and the feature branch not are:

• a mismerge
• a test stability issue
• a functional incompatibility between branches

(I'd note that we only release one feature at a time, so the latter will resolve itself for master to pass).

Answer 1

Unless you think you can't spare the electricity, or the unit tests take an inordinate amount of time (measured in days), I see no reason why you wouldn't go ahead and run the tests on the feature branch first.

The whole point of a CI server is to run those tests so that you don't have to. It's the cheapest form of insurance available to you. Wouldn't it be nice to find out that a problem has crept into your feature branch before you merge into master?

Oh? The odds of finding something at that level are small? How much did you say that electricity costs you again?

Answer 2

Without test result in the target branch, you cannot claim that it is the feature branch development which has broken the test. There are few as demoralising things as having to deal with somebody else's mistakes while you are trying to develop your feature.

Answer 3

The entire purpose of the "develop" branch under GitFlow is to do tedious slow things with the entire codebase on a regular basis for integration purposes. From the original GitFlow description (https://nvie.com/posts/a-successful-git-branching-model/):

We consider origin/develop to be the main branch where the source code of HEAD always reflects a state with the latest delivered development changes for the next release. Some would call this the “integration branch”. This is where any automatic nightly builds are built from. [emphasis added]

You wouldn't do an automatic nightly build without having the CI machine run the test suite first. That's when you are supposed to do it.

Plus, the entire point of CI is to... continuously integrate. If you aren't testing the result of all of those merges, then you're not integrating, you're dumping everything in a giant bucket and praying that it will work when you decide to release.

Answer 4

I would expect to run unit tests for more or less every build, given the structure you describe I'd also expect to be running a suite of integration tests on the testing branch.

This suggests that the question one might ask is why is the unit test suite too slow? (You'd need to define "slow".) From there how can we reduce the time taken to run the test suites.

The key thing is that we don't expect the tests to fail once they hit the build pipeline - but we run them anyway because they are there to catch what we don't expect.