How to implement ownership of microservices while maintaining business agility?

Question

I am struggling to reconcile some good recommendations, relating to the microservice architecture and Agile and DevOps, that are mutually exclusive in my mind.

On one side we have the recommendation that each microservice should have a small, two-pizza team that owns it (e.g. see "Service per team"). This is the model that the company I work for has adopted. There are several benefits to this approach:

• Cognitive load is kept low. (A huge "pro".)

• Teams take pride in maintaining the quality of the services they own.

• Teams are able to enforce that new code complies with their coding standards and design principles.

• Teams know they are unambiguously responsible for the performance of their services in production.

These benefits are evident when you compare the state of our microservices to the sorry state of our monolith, which is owned collectively and therefore by no one. However, there are also a number of challenges and I do not see how to reconcile them within this framework. Namely:

• Complex coordination for any medium-to-large projects. Simple stories can be implemented by one team on their own, but any more complex undertaking ends up spanning multiple services owned by different teams. Teams end up waiting on other teams to finish their bit.

• High WIP. Work must be allocated in order to keep every team busy at any time. Once a team did their part for a larger project, they look for other things to do. They are effectively banned from helping other teams finish the rest of the project, so they take on a new one.

• Narrow value streams. In the idealised implementation of this "service per team" pattern, teams are decoupled and do not require input from other teams for any of their work. Even if this could be realistically executed, you'd end up having as many projects in flight as many teams you have. The WIP across the organisation remains high. Each team is a value stream, and no value stream may be served more than 9-or-so engineers.

• Suboptimal prioritisation. Team availability becomes a key factor for prioritising incoming projects. As a result, projects bringing relatively little benefit to the organisation may be prioritised ahead of projects with a greater potential return.

• It does not scale. Not easily: in order to add a team, you may need to first re-architect an existing service and break it up in two. Even if a team already owns multiple services, and so ownership of some may simply be reassigned to the newly formed team, you still end up increasing handoffs and all the complexity listed in previous points.

Q: Is exclusive ownership of microservices by specific teams a goal worth pursuing at all, in your opinion? How do other organisations manage it? Would that be better for services to be owned collectively by all teams within the same value stream? How do you limit cognitive load experienced by engineers without overly restricting the organisation's ability to assign resources to highest priority projects?

P.S. I realise that above I assumed that the owners of a service would be the only team doing work on it. This may not necessarily be the case: ownership is also intact if other teams submit pull requests for the owners' review. However, review, and potentially testing, is still work that needs to be carried out by the owners. On the other side, teams tend to be reluctant to take on work related to services they have little experience working with.

Answer 1

Although the "service per team" approach is common, if you consider microservices as components to a broader system, you can consider stream-aligned teams, perhaps with platform teams or complicated-subsystem teams as necessary. A stream-aligned team would work across services, instead focusing on a particular use case or user segment. Highly complex services or core services that are used by many different streams may be supported by more specialized teams. Just because you have a microservice architecture for your system doesn't mean that your teams need to follow the same organization.

This resolves many of the concerns with the service-per-team approach, but it still requires coordination. Teams need to be aware of changes across services and may need to coordinate if multiple teams have a need to change the same service at the same time (or very close together).

Inner source techniques can be useful, as well. Although you may have a team own a service or set of services, you can apply open source techniques like issues, discussions, and pull requests to create a community where anyone who follows a development standard can contribute to a service. It mitigates some concerns, but would require someone from the server owner team to review and accept work and conflicting priorities may still get in the way.

You could also take a more extreme approach and eliminate teams. Although I've never used it, I've been seeing more interest in dynamic reteaming and approaches like Fluid Scaling Technology for Agile. If the size of your whole product team isn't too large, you can use approaches to dynamically form and reform teams across the product as a whole. These techniques can spread knowledge throughout a development organization, but would probably be seen as more radical or extreme.

The approach that you take does depend on the organization. Some organizations may be perfectly fine with teams owning services and dealing with the problems. Others may be willing to experiment with alternative structures or practices. There's not going to be a silver bullet, though. No approach is perfect and there's going to be tradeoffs made.

Answer 2

There is no universal right and wrong here. It's a matter of how many resources you're willing to invest and which potential future problems you want to guard against.

The analogy here is like comparing a supercar and a tank. Supercars are highly performant but not resilient to issues, whereas a tank is not highly performant but is relatively resilient to issues.

"Service per team" is a supercar. It's highly performant by keeping the amount of cognitive load for a given service at a minimum, and in everyday development there is little time wastage on needing to manage and communicate to a larger team. However, this system is not resilient to issues(e.g. staff changes, pre-emptively writing documentation, a sudden need for more developers to quickly develop a new feature).
It takes time and effort for a new developer to respond to a sudden need in this team, as it will take time to learn the ropes that they have not already spent learning.

Business agility here is a tank. It's not as efficient because it requires developers to have their hand in more projects. In everyday development, things will run slightly slower because of this. Developers spend more time reading docs of things they don't often use, or getting back up to speed on something they've been away from.
However, when you hit the proverbial pothole, e.g. developers being absent/leaving, sudden spikes in development for features, ... the company is able to almost immediately deploy any of its developers to assist where needed, as most developers have already pre-emptively learned the ropes during their everyday agile development.

There are pros and cons to both. There is no objectively superior answer here. Does your company rather go fast during everyday development and suffer harder through the potholes; or would they rather slow down the everyday development pace but then being able to brush off the occasional pothole without much hassle?

That's a business decision, not a technical one.

The only thing that I'd really add here is that I tend to consider either extreme end of the spectrum to be naive and risky, unless there is conscious and explicit justification for it (and, IMO, an explicit acknowledgement of the risks being taken and acceptance of the responsibility by the decision maker).
In most cases, you'll want to employ a "most of column A, some of column B" approach here, to ensure that neither approach's flaws become a mortal weakness.

Answer 3

It is hard to do glass ball consulting, i.e. say anything meaningful which would be helpful without knowing the organizational structure and the nature of the project(s).

Besides, I can offer my thoughts on your challenges:

Complex coordination for any medium-to-large projects. Simple stories can be implemented by one team on their own, but any more complex undertaking ends up spanning multiple services owned by different teams. Teams end up waiting on other teams to finish their bit

I see no reason why technically one team has to wait on another team. There are ways around this; namely feature toggles, versioning of services, designing services which stay in one way or another backwards compatible, long migration paths for services etc.

But the coordination becomes more complex, agreed.

High WIP. Work must be allocated in order to keep every team busy at any time. Once a team did their part for a larger project, they look for other things to do. They are effectively banned from helping other teams finish the rest of the project, so they take on a new one.

I do not get this one.

How is work strutured here? You have several projects each of which has microservices and each team is responsible for one service in each project? Say Team1 is responsible for Project1-Service1 and on the same time Project2-Service8?

you'd end up having as many projects in flight as many teams you have

This looks like like my guess is correct?

Suboptimal prioritisation.

It does not scale.

the state of our microservices to the sorry state of our monolith

As I've said in the beginning, it is hard to say anything meaningful. But on the otherhand from what you write I get the impression you made a switch from developing a monolith to doing microservices in the hope structures in software would help to cure structural problems in the organization.

From my POV it should go the other way around: you have problems in your software, e.g. increasing complexity making releases harder, resulting in organzational problems, which (hopefully) could be bettered with microservices.

Perhaps it would make sense to first think about organizational structure and how your work could be structured and in a second step look for methodologies which would be a good fit and benefit the way you want to work. Whether Scrum / Microservices are good for you is up to your company.

And in general I see no problems in doing scrum and having microservices as such - perhaps in your context and your way of doing things it may make no sense.