8

There are many claims about existence of clusters of bugs or defects. A simple search reveals multiple results, for example: 1, 2, 3, 4, 5.

However, all the evidence cited is anecdotal and I could not find any concrete data to back this up. While my own experience does not contradict these claims, people love to see patterns even when there is none (even uniform distribution of bugs will produce clusters, and it might be easier to remember when you have to fix 10 bugs in one place rather than 10 unrelated things all over the codebase).

I am genuinely curious if this phenomenon indeed exists, but I could not find any objective or even semi-objective source (as in test, experiment, study, etc.) which would show that defect clustering really does happen.

Of course, I am fine with assuming bug clustering hypothesis as good practice (even if it is false, it won't hurt too much). On the other hand, concrete data could shed some light why it happens. Is it because of those days one has a terrible headache (for whatever reason)? Or maybe because some parts of the code are just hard and others are easy? Or perhaps it is the place of responsibility of those two engineers that don't like each other?

My question: Does defect clustering effect indeed exist? Is there any concrete non-anecdotal data that is best explained by this hypothesis?

  • The reason is because one bug can breed other bugs, this happen because code is interlinked, so while tester feel good finding many bug sometimes they dont know that programmer just need to fix that one bug and poof all the other is gone. – kirie Nov 15 '16 at 16:00
  • I'll agree with @kirie here, that a bug in one piece of functionality usually has a cascade effect on other pieces of functionality. The tester may think they are distinct bugs, but they are really all sourced from the one problem. Additionally, humans are well designed to find patterns, which is why we do it in everything. – Marshall Tigerus Nov 15 '16 at 17:04
  • Very rarely I have a bug that may overwrite a random bit of information, anywhere. With that kind of bug in the source code, the software could misbehave in gazillions of possible ways. – gnasher729 Nov 15 '16 at 22:24
  • 2
    I think this is a valid question and would not like to see it closed, as the OP specifiably asked for "concrete non-anecdotal data". However, the answers given so far are not providing this. I would rather see it protected and answers without links to research down voted. – mattnz Nov 16 '16 at 1:02
  • @gnasher729 I do not know what overwrite bit of information you mention, but this is common when you use DRY principle on early stage when many function havent been fully tested yet already used many times. – kirie Nov 16 '16 at 4:58
3

I don't have any data at hand, but I am pretty sure the clustering hypothesis is true. Best guess of mine are these two cases happen more or less frequently:

  • a piece of code or algorithm is complex (maybe the implementation is more complex than necessary) and the original programmer did not fully understand what his code might do because of the complexity.

  • the code was not tested well

And - of course - a combination of both. Testing is hard, but testing complex code is much harder by an order of magnitude. And with increasing complexity, especially when code is not well tested, in my experience, the number of potential bugs in a piece of code increases disproportionately high.

So if you find several bugs in a given piece of code, it is most probably a badly tested, complex piece of code, which gives you a high chance to find more of them in the same area.

2

Formal studies like this seldom exist in software development, probably because programming (despite its association with machines) is primarily a human endeavor, not a machine one.

Yesterday I was fixing a bug in a SQL statement that involved two SELECT statements and a UNION. Both SELECTs were returning the same result due to a simple error in a JOIN. Fixing the problem, however, uncovered another bug that was being masked by the first bug.

2

In my experience:

Clustering happens when work gets interrupted. Say someone is shifted off the project so his work isn't fully tested, or maybe even completed, and/or the results not fully understood.

Clustering also happens because of the "bad programmer" problem. Say 5 people worked on something and one of them was sub-standard. The bugs will be associated with his work.

The Pareto Principle applies (aka the 80/20 rule). Roughly 80% of the effects come from 20% of the causes. https://en.wikipedia.org/wiki/Pareto_principle Note this observation dates back from before computers.

0

There is no paradox in bug clustering. And our cognitive biases fan the flame around.

According to normal distribution at any given moment of time some parts of the codebase are significantly more buggy than others. Any new bug is more likely to be found in the buggy part.
So the one you are about to fix is already doomed with a good chance to have a company.

It's the same as "misfortunes never come singly".

  • 1
    I'm not sure the normal distribution allows us to make an inference as you're suggesting. I assume we're analysing defect density per code unit. For any non-uniform probability distribution, we can see that some units will be more buggy than others. For symmetric distributions like the normal distribution, exactly half of all modules will have above-average defect density! Of course, that's a consequence of assuming a constant risk of bugs across all units—but isn't that the opposite of what this question is about, that bugs breed more bugs? Maybe I misunderstood this answer. – amon Nov 16 '16 at 18:12
  • "exactly half of..." yes, but it has no value in the current context. Sorry, I did not get understand you Amon. I don't agree with exact phrase "bugs breed more bugs". My point is that a bug found is [with probability we can't ignore] destined to be among others. – Vlad Nov 18 '16 at 16:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.