1

We have multiple developers contributing to a project, using Git as our repository. We have a QA branch that matches our QA environment and a master branch that matches production.

Features can be in QA for weeks, because other more urgent requests come up, or there are issues with a particular feature that take the developer a long time to resolve. Thus, we have a need to release specific features without including other features that are still in QA.

I understand that this is far from ideal, but it would be difficult to make changes to this process, due to budget, organizational contraints, and client demands.

The current process is:

  • The developer creates a feature branch from master.
  • When the feature is ready for QA, they create a pull request into QA.
  • When it has passed QA, management creates a pull request for their feature branch into master.
  • Features are not deployed in any particular order unless they are dependent on each other.
  • After a release, we merge master back into QA.

We use Azure DevOps to create and merge the pull requests and to release to QA and production. I mention this because ideally we would like as much of the functionality as possible to be performed in DevOps and not need to use the command line or Visual Studio to merge branches.

The current process works OK, except for the following issues:

  1. In Azure Devops, when the developer creates the pull request into QA, it often gives a "multiple merge base detected" error and/or shows changes from other pull requests that have been merged into QA but are not in master yet. This is confusing because it doesn't show just the changes that their branch will be introducing.

  2. Because feature branches are created from master, developers do not have access to changes that other developers have already made in QA. This becomes an issue if the change they are working on is dependent on another change that is in QA. They could merge the other developer's feature branch into their branch, but we are looking for a simpler process.

  3. We end up getting a lot of merge conflicts in QA that are not actually conflicts, probably because of the "multiple merge base" issue. This becomes time-consuming and confusing to resolve as the QA and master branch diverge.

We were thinking of changing the process as follows:

  • The developer creates their feature branch from QA.
  • When the feature is ready for QA, they create a pull request into QA (as before).
  • When the feature is ready to release, management cherry picks the pull requests that should be released into master.

It seems like this would solve the problems I mentioned, but after doing some research, it seems like cherry picking is problematic and not recommended except for hotfixes. Also, cherry picking conflicts cannot be resolved in Azure DevOps, unlike regular merge conflicts, which is not ideal for us.

Which of the two processes I described would work better for our organization? Or is there another branching and merging model that would work better for us?

I really appreciate any advice and help you can provide! I welcome advice on changes we can make to the development process, but this would probably not be possible any time soon. I would really like to undertand how we can best support the current process.

EDIT: It seems like feature flags are a popular solution. I don't think this will work with our current architecture, because it would require flags to be spread throughout the code base and introduce complexity. Due to several constraints, we are unable to do any major refactoring right now.

A couple people suggested testing each feature branch in isolation, and then merging into master when QA is complete. This might work for us, except in the case where multiple features need to be tested together.

Thanks to @ThomasOwens for linking to Martin Fowler's article on branching patterns. I read it, and it sounds like the alternative branching strategy I proposed in my original post is very similar to the Maturity Branch pattern. In that pattern, he shows individual commits being cherry picked to a release branch. Maturity Branch diagram

In my proposed branching strategy, our QA branch is the "mainline", our staging branch (which I didn't mention before) is the "release branch", and master is the "production branch".

Anyone have any feedback on this branching pattern? Pros / cons?

10
  • I would ditch the QA branch and tell the QA guys to checkout the respective branch they want to test. If you're using Github or Gitlab I'd also recommend making use of the code-review and approving features there. Gitlab also supports kanban boards where you can swap issues (and attached MRs) between different lanes (phases or tags) and thus tell your QA team that certain features are ready for various acceptance tests Dec 15, 2023 at 2:43
  • 1
    I would ditch using feature branches at all, and instead use feature toggles
    – Doc Brown
    Dec 15, 2023 at 7:14
  • 1
    Does this answer your question? Git branching strategy for long-running unreleased code
    – Doc Brown
    Dec 15, 2023 at 7:14
  • Roman, thanks for the suggestion! That's similar to what Thomas recommended with having separate QA environments per branch. It would require the QA team to have Visual Studio and it may be a little technical for them, but it's a possible option. As I asked him, what happens if you need to test multiple features together? Create an integration branch and just put those features into it?
    – Lisa S
    Dec 16, 2023 at 1:23
  • Doc, feature toggles would be too complex with our current architecture, and there is zero chance of changing it in the near future. Does anyone have feedback on the two branching models in my original post and what the pros and cons are?
    – Lisa S
    Dec 16, 2023 at 1:27

1 Answer 1

5

No branching strategy will help you much if your features can be in QA testing for weeks. You need to make investments to reduce that time and complete QA faster. This often takes the form of extensive automation testing and the creation of build pipelines to run those automated tests frequently.

If you can instantiate multiple QA environments, a branching strategy closer to gitflow may help. Instead of having branches per environment, you can deploy your feature branches to your QA environment and then merge them to the development branch after they pass. However, you'd still need to keep your feature branches up-to-date with the development branch and that changes from the development branch don't cause issues with the feature.

A better solution would be to use trunk based development with feature flags or feature toggles and/or keystone interfaces. Using feature flags and keystone interfaces does require architectural support. Without designing with this in mind, you may have a system that requires your flags to be scattered throughout the system and would incur a higher risk for a mistake (such as including too much or too little behind a flag) or the added complexity of dealing with flags. This would allow you to enable and disable features in different environments yet keep the codebase the same. Once features are well-tested, you can turn them on in production.

If you want more ideas, perhaps check out Martin Fowler's writing on branching patterns. What you are doing now is close to environment branching, which is considered an antipattern. You may have better luck with what he calls the release train, for example. You can read this article with the team to look at common problems and solutions.

7
  • Thank you for your response! I don't think feature flags would work for us with the current architecture; as you point out, they would have to be scattered everywhere. Thanks for the link to Martin Fowler's branching patterns, I will definitely study this in detail. Release train is a possible option. I also think having multiple QA environments and testing each feature independently might work. In that case, developers would create the branch from master, make changes, branch gets tested, and we merge back to master? What if there are multiple features that need to be tested together?
    – Lisa S
    Dec 16, 2023 at 1:02
  • Thomas, I have a question about release train. It sounds like developers create their feature branch from master and create a pull request into the release train branch. Is that accurate? If so, it seems like the same issue that we are facing currently with "multiple merge base detected", and with changes they didn't make showing up in their pull request would occur. I am fairly new to Git so I'm not sure if there is a simple solution to this issue.
    – Lisa S
    Dec 16, 2023 at 1:44
  • @LisaS If you're using multiple QA environments and gitflow, you branch from develop, deploy a feature branch to QA, and merge to develop after approved. You would need to possible retest some aspects after merging develop into your feature branches or merging a feature branch into develop, depending on the timing and the changes. You can also retest in your release branch when that's created. This is why investing in test automation and automating in parallel with feature development is crucial - it reduces the time to test and can even eliminate most, if not all, manual regression testing.
    – Thomas Owens
    Dec 16, 2023 at 2:08
  • @LisaS Unfortunately, I've never used to release train branching strategy. I know it exists and is described in the article, so I mentioned it since it's applicable to heavier, more formal release processes. Most of my experience is in gitflow, a variation of gitflow, or trunk-based development.
    – Thomas Owens
    Dec 16, 2023 at 2:10
  • This is a good answer, still I think your concerns agains feature flags are exaggerative. To prevent feature flags to be scattered throughout the system you often don't need "architectural support", you just need a single (or maybe a few numbers) of "feature entry points" for a certain feature, and make it a rule in your team to query the feature flags at those entry points exclusively. In my experience, that works for 95% of all features (but YMMV).
    – Doc Brown
    Dec 16, 2023 at 9:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.