Rapid rate of merging new code with minimal testing [closed]

Question

In some companies, code is merged at a rapid rate by multiple teams with minimal testing and somehow the number of bugs is kept to a minimum.

In other companies, this leads to chaos and terrible situations. Root-causing leads to no useful insights and people feel they are put on the spot about a problem (chaos) that is outside of their control.

What is the key difference between these two kinds of companies, ones that can thrive with rapid coding and merging versus those that cannot?

Put another way: What are the failure modes of rapid code merges and what can protect against such failures?

Answer 1

There are a lot of things that can be the difference between a smoothly operating team and chaos.

• Enough testing is done to ensure a working project, and it's automated so it's not apparent that is happening.
• Companies have review gates with knowledgeable enough devs reviewing that major bugs are more frequently identified. Also more likely in smaller codebases.
• Developers have a higher sense of ownership/domain knowledge so they are able to both develop changes faster and more correctly because they understand the domain.
• Well designed contracts make code easier to write, less likely to have surprise side effects lead to bugs.
• Proper build tooling around managing dependencies and preventing conflicting versions of libraries.
• A culture of collaboration and ownership vs one of blame and punishment, formal RCA process tends to be about blame not learning how to avoid problems.
• Developers are just better/more experienced so less mistakes happen.
• Well understood requirements are easier to correctly implement.
• Build systems that can revert to known good deploys, and deploy quickly and efficiently. Bugs are a lot less noticeable if a developer can identify a problem and revert or fix in minutes instead of hours or days. If some part of a deploy is manual (commonly databases) then problems are for more likely.
• Good development or test environments that match production closely enough that issues can be identified with minimal testing early on instead of only being able to verify in production.
• Well designed and architected systems, type systems can catch many types of errors at build time if designed to, internals are hidden and exposed in ways that avoid improper use.
• Lack of a sufficient user base to identify bugs in the first place.

Answer 2

I'm going to challenge your premise. In my experience merge problems are a thing of the past and not common in a modern company.

Merge problems are common if.

• You are using some source control other than git. If you have perforce style branching for example
• You have a monolith application
• You have multiple big, unrelated, features being developed at the same time
• Developers are working by themselves rather than as a team

30 years ago this would be the standard way to develop code and yes, you would get screwed over when some other dev merged their feature before yours. But these days everyone is:

• Using git, which has a much more forgiving branching method
• Working together in small agile teams
• Doing small features and committing often
• Splitting software out into microservices and libraries
• Using continuous integration to prevent broken builds being merged

This way of working has the side effect of reducing or preventing "the big merge"