Bug clustering in source code

Question

There are many claims about existence of clusters of bugs or defects. A simple search reveals multiple results, for example: 1, 2, 3, 4, 5.

However, all the evidence cited is anecdotal and I could not find any concrete data to back this up. While my own experience does not contradict these claims, people love to see patterns even when there is none (even uniform distribution of bugs will produce clusters, and it might be easier to remember when you have to fix 10 bugs in one place rather than 10 unrelated things all over the codebase).

I am genuinely curious if this phenomenon indeed exists, but I could not find any objective or even semi-objective source (as in test, experiment, study, etc.) which would show that defect clustering really does happen.

Of course, I am fine with assuming bug clustering hypothesis as good practice (even if it is false, it won't hurt too much). On the other hand, concrete data could shed some light why it happens. Is it because of those days one has a terrible headache (for whatever reason)? Or maybe because some parts of the code are just hard and others are easy? Or perhaps it is the place of responsibility of those two engineers that don't like each other?

My question: Does defect clustering effect indeed exist? Is there any concrete non-anecdotal data that is best explained by this hypothesis?

Answer 1

I don't have any data at hand, but I am pretty sure the clustering hypothesis is true. Best guess of mine are these two cases happen more or less frequently:

• a piece of code or algorithm is complex (maybe the implementation is more complex than necessary) and the original programmer did not fully understand what his code might do because of the complexity.

• the code was not tested well

And - of course - a combination of both. Testing is hard, but testing complex code is much harder by an order of magnitude. And with increasing complexity, especially when code is not well tested, in my experience, the number of potential bugs in a piece of code increases disproportionately high.

So if you find several bugs in a given piece of code, it is most probably a badly tested, complex piece of code, which gives you a high chance to find more of them in the same area.

Answer 2

Formal studies like this seldom exist in software development, probably because programming (despite its association with machines) is primarily a human endeavor, not a machine one.

Yesterday I was fixing a bug in a SQL statement that involved two SELECT statements and a UNION. Both SELECTs were returning the same result due to a simple error in a JOIN. Fixing the problem, however, uncovered another bug that was being masked by the first bug.

Answer 3

In my experience:

Clustering happens when work gets interrupted. Say someone is shifted off the project so his work isn't fully tested, or maybe even completed, and/or the results not fully understood.

Clustering also happens because of the "bad programmer" problem. Say 5 people worked on something and one of them was sub-standard. The bugs will be associated with his work.

The Pareto Principle applies (aka the 80/20 rule). Roughly 80% of the effects come from 20% of the causes. https://en.wikipedia.org/wiki/Pareto_principle Note this observation dates back from before computers.

Answer 4

There is no paradox in bug clustering. And our cognitive biases fan the flame around.

According to normal distribution at any given moment of time some parts of the codebase are significantly more buggy than others. Any new bug is more likely to be found in the buggy part.
So the one you are about to fix is already doomed with a good chance to have a company.

It's the same as "misfortunes never come singly".