20

We are a big team (10-12 developers and 4 qa) working on multiple projects with the same git repository. Its a spring boot based backend web service. We are looking for a good git branching and deploy strategy. we also have a qa team who ensures that our features are working as expected (bug free to a certain extent).

After reading few articles, I got a feeling that the Gitflow model would work well for us. Here comes my question.

Where should our QA team test our features ?

  1. should they test on feature branch, where they will raise bugs and developer will fix it and once it passes QA test we merge to develop. and QA will again do the integeration testing in develop branch.
  2. should we merge all the features (after unit tests and basic testing from developer) to develop branch and let the qa test from there. fixes and tests will all happen in develop as well.

I am curious to hear what approach worked well for others.

13

QA should probably be testing twice.

The first testing should be around the specific changes and done on the feature branch. This lets QA test around the specific changes and see that the particular change is complete as specified and behaves as expected. It also gives them an early preview for the second round of testing, which is what actually matters for QA.

Coming from my background in various regulated environments, the second testing needs to be done on either a tag on the development branch that corresponds to a release, the the release branch, or perhaps the master branch. Prior to a release, QA should be testing the full and complete code that is going to be deployed prior to it being deployed and you should be able to assert that whatever was tested by QA is exactly identical to what gets deployed to production. My preference would be that after a code freeze, a tag is applied to the release branch and QA would test that. Changes would be done in a develop branch, spot checked, and then tested again in a new tag on the release branch. These tags on the release branch would correspond to release candidates. An approved release candidate would be merged into master and then identified as the released version.

I am making a few assumptions here. First, you have somewhat decent developer-based test coverage. Ideally, this would be automated unit and integration tests that developers run and do so before sending any code on any branch to QA. Developers may also want to do some exploratory testing around the UI to make sure that things look right before QA testing. Second, there is good coordination between development and QA to plan the changes being incorporated to ensure sufficient QA time based on features.

  • 2
    few concerns i have with this approach are 1) every feature would require a machine to deploy on. sometimes we work on 5 features some times only couple. may be we can setup jenkins to spin up VM's? what does everyone do ? 2) qa needs to know which build is on which machine. 3) i wondered whether whether its redundant as we are going to do thorough testing anyway in release branch. – srini Mar 20 '17 at 3:40
4

Great question. I don't think there is an 'official' correct answer to this. It depends on how fast you can test.

The essential problem is that each merge, compilation or even deployment, can potentially create a bug. The further 'up' the chain you test, the sooner you know about bugs, but also the more times you have to re-test.

In order to be assured that you have tested the software the customers are using you really have to test the live deployment before the customers traffic (assuming a web app) is routed to those servers via a blue/green deployment pattern.

But obviously this is a bit late in the day to be the first time you checked the code at all!

If you test a release branch in a qa env then you can take the risk and reduce the live testing to smoke tests only. and apply bug fixes to the release branch. But you cant assess whether a feature is complete before creating a release

If you test development then you get mini bug-fix-feature branches. Features are still merged before they are assessed, plus features for the next release can collide with testing the current release.

If you test Feature branches you need a million environments and have to orchestrate the order of merges and test sign offs. plus a lot of retesting.

From my experience I would recommend:

quick test of feature branch on dev machine. just ensure its feature complete and the testers/devs agree on what the requirements mean.

Day to day testing/automated testing on dev branch deployed to qa servers. Lets you test all the features together and say when you are ready to release.

If all the features are in but testing isnt finished. eg the sprint is complete. make a release branch and deploy to a second qa environment. This allows for bug fixing/testing on release 1 to continue at the same time as features for release 2.

(scrum devotees will say you should put only bug fixes into sprint 2 but lets be practical)

Smoke tests on live green/blue deployment before switch over. These are super important as you will pick up config/environmental errors that no-one really looks for while developing.

-2

I agree with Thomas Owens. You should probably be testing twice. Once on the feature branch before it is merged and once on your main branch before you release.

In fact, I love that workflow so much I made a tool, Topico, which automatically builds and runs disposable versions of your app for each pull request, each with its own unique testing URL. This allows your QA team to test feature branches in isolation without the need for some kind of dynamic test environment set up on their own machine.

This approach will mean only code that has passed human testing will ever reach your main branch, thus maintaining its integrity.

I introduced this at a couple of companies and it helped our release cycles a lot. We are now able to accurately schedule releases, and we're way better at understanding what is likely to make it into the next release and what will have to wait for a future release. It just gives you a lot more confidence.

  • I can only assume the downvote was because someone took offence to me mentioning my own tool. This tool specifically addresses the concerns the OP's expressed in the comments of Thomas Owen's answer so I'm not sure the downvote was warranted. – nlyn May 1 '18 at 15:39

protected by gnat Apr 30 '18 at 18:17

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.