Managing QA testing when there's pressure to merge things at the earliest possible

Question

I'm working in a team of 12 where we have around 8 developers and a QA. We do not follow a git flow model but we do have feature branches where individual feature developments take place and then these are merged to master and then released. As there are many developers and only one QA, the QA person always creates a bottleneck as testing has to be done on feature branches before merging to master.

But this blocks the developers as merge conflicts occur due to the time taken during the testing phase. Therefore the QA person has to be asked to merge things at the earliest possible and then testing everything together in master.

Is there anything that could be done to refine this process? Will following a git flow help? Though it adds an additional layer of development branch which could cause more maintenance worries?

Answer 1

I would do QA on the dev (or master in your case) branch rather than the feature branch.

Devs should test on the feature branch to ensure the features are complete, but QA should test the product as a whole after merging in the new feature.

Otherwise you are testing twice, on the feature branch and after merging

Just to do the sums, assuming no bugs found

Say you have a sprint with 3 features which are completed on day 1, 2 and 3 of the sprint

Testing on feature branch - no merge backs from master

• test FB1
• test FB2
• test FB3
• merge all
• test master
• total test runs : 4

Testing on feature branch - with merge backs

• test FB1
• test FB2
• test FB3
• merge FB1
• test FB2
• test FB3
• merge FB2
• test FB3
• merge FB3
• test master
• total test runs : 7

Testing on Master only

• merge all
• test master
• total test runs : 1

Answer 2

Putting the dev/tester ratio issue aside, since you probably aren't in a position to fix that, the problem seems to be with your feature branches lagging behind master.

As Ewan has pointed out, you want to be testing code that is representative of what will be deployed. So, testing on a feature branch that is behind master is not great.

What you could do is make sure your feature branches stay up-to-date with master by rebasing them every time master is updated.

Yes, it creates more work, but it would allow you to keep your testing in a feature branch, while ensuring that you're testing the code as it would be in production. And it would ensure that the merge to master after the QA pass remains easy.

The only thing left is to decide whether it is the Dev or QA's responsibility to rebase the feature branches. I would suggest the Dev since they would know best how to take on changes from master without affecting the new feature.

Answer 3

Stop asking QA to test everything all the time. Get the developers to do it themselves.

Where I work, QA don't do testing. They are there to ensure that developers are following the correct process.

The only time they get involved in testing is the final formal test before software is released to a customer. Even then, they are there to ensure that the test is being carried out correctly, not to actually press the buttons.

Answer 4

A few things stand out to me.

First, the ratio of developers to testers stands out as being very lopsided.

In my experiences, a good ratio is about 3-5 developers to 1 manual tester. This seems to work out well as long as the tester is involved from early in requirements development to begin to start identifying test strategies, approaches, and cases. The work to develop the tests begins at the same time as development on the functionality. In cases where the development is done and the work is handed off, I've found that the developers need to shift away to support testing as the tester has questions or isn't sure about behavior in edge cases unless it's explicitly documented somewhere.

Second, the testing in feature branches before merging to master is concerning. This is increased since there are conflicts that need to be resolved when merging the feature branch into master.

It seems like the code that is being tested is not the code that ends up in the mainline of the product. This means that you can't be sure that your tests are finding the problems that exist in the post-merged product. There are two ways to correct this. First, regularly merge the master branch into long-running feature branches and test. You may not need to retest everything, but if there are conflicts, you can analyze the diffs and choose what tests must be re-executed. Second, if you are working on a planned release cycle, something like the gitflow model may be useful in ensuring that master reflects releases while having feature and development branches that are synchronized with each other.

Third, automate.

Work on automating your test suite. Make it very easy to run various test suites. A good focus would be on regression - finding and ensuring that previously fixed issues don't return. But automate as much as possible from the unit through the system level and make it easy to run. Integrating into a build process would also help to ensure that each branch is in a good state.

Fourth, quality is everyone's business.

Shift as much of quality assurance to everyone on the team. This may not be possible - in some cases, there are contractual or regulatory requirements that force a separation between developers and testers. Even in these cases, try to gain a full understanding of what the requirements really are and try to share the workload. From my experiences in regulated environments, organizations tend to misinterpret requirements, including around verification, validation, and independence and it's often not necessary to always have a hard line between development and test, and even when it is, development can do some level of testing to ensure that the testing group has a much easier time of it.