40

We are a big team (10-12 developers and 4 qa) working on multiple projects with the same git repository. Its a spring boot based backend web service. We are looking for a good git branching and deploy strategy. we also have a qa team who ensures that our features are working as expected (bug free to a certain extent).

After reading few articles, I got a feeling that the Gitflow model would work well for us. Here comes my question.

Where should our QA team test our features ?

  1. should they test on feature branch, where they will raise bugs and developer will fix it and once it passes QA test we merge to develop. and QA will again do the integeration testing in develop branch.
  2. should we merge all the features (after unit tests and basic testing from developer) to develop branch and let the qa test from there. fixes and tests will all happen in develop as well.

I am curious to hear what approach worked well for others.

Improve this question

2 Answers 2

Reset to default
27

QA should probably be testing twice.

The first testing should be around the specific changes and done on the feature branch. This lets QA test around the specific changes and see that the particular change is complete as specified and behaves as expected. It also gives them an early preview for the second round of testing, which is what actually matters for QA.

Coming from my background in various regulated environments, the second testing needs to be done on either a tag on the development branch that corresponds to a release, or the release branch, or perhaps the master branch. Prior to a release, QA should be testing the full and complete code that is going to be deployed prior to it being deployed and you should be able to assert that whatever was tested by QA is exactly identical to what gets deployed to production. My preference would be that after a code freeze, a tag is applied to the release branch and QA would test that. Changes would be done in a develop branch, spot checked, and then tested again in a new tag on the release branch. These tags on the release branch would correspond to release candidates. An approved release candidate would be merged into master and then identified as the released version.

I am making a few assumptions here. First, you have somewhat decent developer-based test coverage. Ideally, this would be automated unit and integration tests that developers run and do so before sending any code on any branch to QA. Developers may also want to do some exploratory testing around the UI to make sure that things look right before QA testing. Second, there is good coordination between development and QA to plan the changes being incorporated to ensure sufficient QA time based on features.

Improve this answer
1
  • 5
    few concerns i have with this approach are 1) every feature would require a machine to deploy on. sometimes we work on 5 features some times only couple. may be we can setup jenkins to spin up VM's? what does everyone do ? 2) qa needs to know which build is on which machine. 3) i wondered whether whether its redundant as we are going to do thorough testing anyway in release branch.
    – srini
    Mar 20, 2017 at 3:40
11

Great question. I don't think there is an 'official' correct answer to this. It depends on how fast you can test.

The essential problem is that each merge, compilation or even deployment, can potentially create a bug. The further 'up' the chain you test, the sooner you know about bugs, but also the more times you have to re-test.

In order to be assured that you have tested the software the customers are using you really have to test the live deployment before the customers traffic (assuming a web app) is routed to those servers via a blue/green deployment pattern.

But obviously this is a bit late in the day to be the first time you checked the code at all!

If you test a release branch in a qa env then you can take the risk and reduce the live testing to smoke tests only. and apply bug fixes to the release branch. But you cant assess whether a feature is complete before creating a release

If you test development then you get mini bug-fix-feature branches. Features are still merged before they are assessed, plus features for the next release can collide with testing the current release.

If you test Feature branches you need a million environments and have to orchestrate the order of merges and test sign offs. plus a lot of retesting.

From my experience I would recommend:

quick test of feature branch on dev machine. just ensure its feature complete and the testers/devs agree on what the requirements mean.

Day to day testing/automated testing on dev branch deployed to qa servers. Lets you test all the features together and say when you are ready to release.

If all the features are in but testing isnt finished. eg the sprint is complete. make a release branch and deploy to a second qa environment. This allows for bug fixing/testing on release 1 to continue at the same time as features for release 2.

(scrum devotees will say you should put only bug fixes into sprint 2 but lets be practical)

Smoke tests on live green/blue deployment before switch over. These are super important as you will pick up config/environmental errors that no-one really looks for while developing.

Improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.