0

We have a suite of integration tests that take about 2 hours to complete.

Currently our flow is that each feature branch runs the integration tests, and only if the branch is updated from master and the tests completed, it can be merged to master. So naturally, it happens quite a lot that you start the tests when your branch is updated, but by the time it ends, there are new changes to master, and you must update your branch and re-run the tests...

We are obviously doing something wrong. But we are not sure what.

Should we allow to merge the feature branch even without updating and testing? it feels risky...

Should we allow to merge, but have another run on master before deployment to production?

Improve this question
1
  • There are CI tools such as bors that do this the right way: (1) Tests pass, feature is ready for merging. (2) Bors performs the merge but initially as a separate staging branch, and runs the test suite again. (3) If it passes, bors updates the main branch to point to the merge commit. No one merges into the main branch directly. Multiple merges can be batched together to speed things up. This workflow guarantees that the tip of the main branch always passed the test suite. A similar workflow can also be done manually as a kind of “release train”.
    – amon
    Commented Feb 25, 2022 at 13:41

2 Answers 2

Reset to default
2

When someone else merges into master whilst in parallel integration tests are running, they can foresee what will happen (in case they get to know about the running tests). Hence, I think the most straightforward solution here is to:

  • ask everyone else to defer merges to master during a feature integration is in progress

  • if an intended merge-to-master is so urgent it cannot wait for two hours, then it is most probably justified to stop the running integration tests from the planned feature integration, and restart them from beginning after the high-priority merge was applied. Ideally, this just happens rarely for emergency situations.

As long as there are not multiple teams in parallel trying to merge half a dozen different features per day into the product, this might be feasible.

Of course, it depends heavily on the team size, structure, merge frequency and working environment how you organize the deferred merge. For some teams, an informal way might be sufficient. Others may require some automatisms which block the access to the master branch during an integration test run.

Improve this answer
1

I would not say that you are obviously doing something wrong, but your current policy does get into the way with the frequency/speed at which you/your team wants to complete their features.

To improve that situation, you will need to do a risk assessment of the possible improvements and find a balance between the increased speed at which features get merged into master and the increased risk that a bug makes it into master.

To help with that risk assessment, it is good to collect some metrics, like

  • How often does the test suite fail when executed the first time after the developer has declared an intention to merge the feature into master?
  • How often are integration issues found with subsequent updates from master?

As you noted already, you can mitigate the risk of merging a bug into master by requiring a successful run of the tests before putting a commit from master into production.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.