Release Management best practice for cross-functional teams

Question

I want to know about best practices for release management of a product-technical team with about 7 cross-functional teams included. We changed our team structure to something like Spotify Squads Framework with some changes because of our team culture and size. now we have 7 cross-functional teams that have all the necessary people (developer, designer, PM, QA) for product development. It has its own agile sessions and delivers the features.

Everything is fine except the release process, we are in microservice architecture and we have many source codes that many of which are shared between two or more teams. each team needs a stage for showing its deliverable features to its product manager and testing process with QA before the production release, but we have only three stages (dev, stage, production) bases on Git Feature Branching Flow.

The questions are:

1. Which team's Pull Request must be merged first in the same situation between two teams?
2. If a PR merged and in the testing process we found problems, what can we do? for this problem we are not able to merge two features at the same time.
3. Most of the time, the development of the backend of a feature is one sprint earlier and clients like to connect to a live stage to test their APIs so we need to merge this feature's backend into develop branch and release the backend on developing environment for using clients. If backend changes have affected other clients, we must wait till this features client development finished and release them at the same time that takes a lot of time. the other teams will be stopped for one team and pull requests stuck in a queue.

Answer 1

Microservices are by definition independently releasable. As Dave Farley says You can have "E2E Testing Strategies" or "Microservices" but not both..

Your "many source codes that many of which are shared between two or more teams" also sounds likely to cause serious problems. Who owns these source code modules? What's the release strategy for them? Unless they are very general modules then it sounds likely these also introduce a lot of coupling between teams where each team can't work at it's own chosen pace.

I would suggest looking to move away from the git branching model you link to - as its author says "If your team is doing continuous delivery of software, I would suggest to adopt a much simpler workflow". Personally I would suggest looking at some form of trunk based development. You only really need one permanent branch, plus a short lived (<24 hours) temporary branch for each developer to do one piece of work.

If features need to be shown to people before release you can either do that by asking them to view it on a developer workstation, or you can use feature toggles to enable the feature only on certain environments or for certain users even while merging it into your main branch.

For now since you clearly have a lot of dependencies between teams and it seems they can't truly release independently it might be best to merge the code into a smaller number of git repositories, so that matching versions will always be deployed together. You could consider separating them out again in future if and when you've eliminated or nearly eliminated shared code and dependencies on the details between one service and another.

Answer 2

You can do some technical trickery by releasing multiple versions of your APIs and routing traffic to the version you want. But I think I would advise a simpler solution. Agree a release schedule with what features are going in each release.

So you have seven products all on the go at once and they all need features across various microservices, you have to get together and say, "OK for the October release for microservice X my team needs feature A what do you other teams need?"

You can then all get on programming the features and merging with each other. You can deploy to a single dev and test environment and demo etc.

It might not be super sexy 10 releases a day coolness, but it will promote cross team cooperation, understanding of the different features and "ownership" as well as giving you more certainty in your delivery dates