I am making 4-5x more story points than average, but producing bugs at half the rate. Graphs say it's 2x more bugs, how to deal with that?

Question

So it is generally accepted that top tier programmers can produce an order of magnitude more/better code than their more average peers.

It's also generally accepted that the rate of errors made in code is relatively constant for programmers.

Instead, it tends to be impacted by the processes used when writing the code and after the code is written. (As I understand it) Humans tend to make mistakes at a fairly constant rate - better programmers just notice more of them and are quicker to fix them.

• Note that both of above assertions come from Code Complete by Steve McConnell - so it's not a matter of differing perspectives.

So I've started to see this recently in my code. I can hammer out about 4-5x the amount of code as many of my peers (measured by story points estimated by the team), with higher quality (based on performance metrics and number of changes made after check-in). But I still make mistakes. Between better unit tests, a better understanding of what the code is doing, and a better eye for issues when doing code reviews I'm not producing 4-5x the number of bugs.

But I'm still producing about twice as many bugs found by QA as other developers on my team. As you might imagine, this causes some problems with non-technical folks doing metric measurements (read: my boss).

I've tried to point out that I'm producing bugs at half the rate of my peers (and fix twice as many), but it's a hard sell when there's graphs saying I produce twice as many bugs.

So, how to deal with the fact that increased productivity will lead to an increased number of bugs?

Answer 1

I think you're mixing your concerns. And there's nothing on your side that you need to change.

Productivity is a hint at how quickly a project will be completed. Project managers and everybody else like to know when the project will deliver. Higher or faster productivity means we'll see the project deliver sooner.

Rate of bugs isn't tied to productivity but rather to the size of the project. For example, you may have N bugs per Y lines of code. There is nothing within that metric that says (or cares!) how quickly those lines of code are written.

To tie that together, if you have higher productivity, yes, you'll "see" the bugs being written more quickly. But you were going to have that number of bugs anyway since it's tied to the size of the project.

If anything, higher productivity means you'll have more time at the end of the project to hunt those bugs down or the developer will be faster in finding the bugs they created.¹

---

To address the more personal aspects of your question.

If your boss is looking strictly at the number of bugs you produce as opposed to the rate of bugs you produce, an educational session is in order. Number of bugs created is meaningless without a backing rate.

To take that example to the extreme, please tell your boss I want double your salary. Why? I have created absolutely no bugs on your project and I am therefore a much superior programmer than you. What? He's going to have a problem that I haven't produced a single line of code to benefit your project? Ah. Now we have understanding of why rate is important.

It sounds like your team has the metrics to evaluate bugs per story point. If nothing else, it's better than being measured by raw number of bugs created. Your best developers should be creating more bugs because they're writing more code. Have your boss throw out that graph or at least throw another series behind it showing how many story points (or whatever business value you measure) alongside the number of bugs. That graph will tell a more accurate story.

---

¹ _{This particular comment has attracted far more attention than it was intended to. So let's be a bit pedantic (surprise, I know) and reset our focus on this question.}

_{The root of this question is about a manager looking at the wrong thing(s). They are looking at raw bug totals when they should be looking at generation rate versus number of tasks completed. Let's not obsess over measuring against "lines of code" or story points or complexity or whatever. That's not the question at hand and those worries distract us from the more important question.}

_{As laid out in the links by the OP, you can predict a certain number of bugs in a project purely by the size of the project alone. Yes, you can reduce this number of bugs through different development and testing techniques. Again, that wasn't the point of this question. To understand this question, we need to accept that for a given size project and development methodology, we'll see a given number of bugs once development is "complete."}

_{So let's finally get back to this comment that a few completely misunderstood. If you assign comparably sized tasks to two developers, the developer with a higher rate of productivity will complete their task before the other. The more productive developer will therefore have more time available at the end of the development window. That "extra time" (as compared to the other developer) can be used for other tasks such as working on the defects that will percolate through a standard development process.}

_{We have to take the OP at their word that they are more productive than other developers. Nothing within those claims implies that the OP or other more productive developers are being slipshod in their work. Pointing out that there would be less bugs if they spent more time on the feature or suggesting that debugging isn't part of this development time misses what has been asked. Some developers are faster than others and produce comparable or better quality work. Again, see the links that the OP lays out in their question.}

Answer 2

Spend some of that extra time cleaning, polishing, and testing your code. There will still be mistakes, but there will be less. That takes time. Your code output rate will go down, but your bug-free code output will increase, and that will result in better productivity. Because bugs are expensive.

Can you keep your code in a branch or a test environment until you kick it around and catch more of the bugs? Bugs in a branch generally cause less waves than bugs in production code.

But I'm not exactly digging your asserts leading into your question. And perhaps your boss isn't either.

I don't think that every coder produces the same rate of errors. Your second link is actually entirely off topic as it's comparing different languages, not different coder skill levels. Code complete mentions some large-sample measurements that are looking at the process rather than the skill of the coders. And when they say that top-tier coders produce more/better code, part of that means that it has less bugs. Depends on the application. So, yeah, I think it IS a matter of differing perspective.

In the end though, if the boss wants cleaner code, give him cleaner code.

Answer 3

I'll go out on a limb and be the devil's advocate. That's not to say I don't sympathise with your plight but, well, my sympathy won't help you. So allow me to add to Philip's perspective:

Your boss cares about the quality of the product, partly because his or her name and reputation will be tied to it. Part of the quality is the perceived amount of bugs. If you sell an awesome drill that drills four times faster than any competing drills, but also breaks down twice as often, you'll have a bad reputation. Even if it's arguable that the drill performs better, people get used to the speed, but they'll remember the breakdowns.

In hindsight, most bugs are obviously preventable. If only you were a little more careful, your boss will feel, you could avoid these issues and any necessary clean-up. From his perspective, that's a trivial, sensical thing to ask, and any arguing you do about it is both not going to work and is going to make you look bad.

He can't measure your superior productivity. You claim higher productivity based on a verifiable metric, so how do your colleagues feel about it? Do they agree, perhaps grudgingly, that you are a more productive programmer, or are you alone in your opinion? You'll make a stronger point if you have other people to back up your assertions.

That's for perspective. Now, how do you go about 'fixing' this frustrating situation you are in?

Do slow down a bit. Mention explicitly to whoever distributes tasks that you're going to try to lower the bug rate (*), so that they're not surprised by your lower intake. If anything, slowing down will reduce the number of bugs assigned to you out of sheer lack of supply.

_{(*) There's a difference between, on the one hand, acknowledging that there are bugs to your name and that you'll try to lessen that amount and, on the other hand, admitting that you have too many bugs to your name and should take action.}

Don't try to convince your boss, because it won't work. Again, that doesn't mean you have to concede your point; you can present a counterpoint and hold your opinion without dismissing his concerns. Because if you do argue the point and cannot satisfactorily prove your superior productivity (and even if you can), you're going to risk insulting your colleagues, or appearing dismissive of them. You don't want to be that guy.

Answer 4

Assuming you would produce the same "amount" of code like your colleagues in 20% of their time, you could spend 4 times as much on really debugging the code and making it perfect, which would reduce your bug rate even more. Then you could call yourself a good programmer.

The biggest question is why you are coding 5 times as much as the others instead of aiming for quality. Is this something your ego dictates you or does your boss force you?

Also you need to consider costs for fixing a bug. When you find it early it you might still be "inside" the code enough to fix it quickly. If it appears only after another month of development, it could be hard to find or even force you to change big parts of the already programmed code.

You seem to have the skillset to produce code, and you could make it awesome if you put your focus on quality instead of speed. Try it a while, my guess is you will like it.

The next problem then is to get the acknowledgement (speak money) for your better performance. Talk to your boss and ask him how to proceed, it is his company and money after all, and if he wants you to produce less bugs, he should also decide in which way it happens.

Answer 5

Developers can be bright, even genius, with an aptitude for understanding and coding solo, without being GOOD developers. A good developer creates a quality piece of work, and makes the whole project better.

I'm not saying this is you, but one of the most frustrating programmers I ever worked with wrote more code than anyone on the team, and we had good people on the team. We tracked commits with a ranking software, so it was almost a competition for some people. He churned out swaths of code, leaving behind him ZERO documentation, impenetrable forests, and actually made some of my own code hard for me to understand (I can do that on my own, thank you very much!). Eventually he almost derailed the project, because he became a one man show. People could not follow him. We were not in synch as a team. We actually rewrote some of his features years later, just to regain maintainability.

The thing I wanted from him was to slow down, and spend more time: 1) Improving the quality of the codebase 2) Communicating with the team 3) Working on things that helped others as well as help him finish features / stories 4) Fixing bugs

I don't agree with measuring productivity by lines of code, but it is a key metric. Does your code counter include comments? If so, there is an easy way to maintain your line output while reducing your "bug ratio"; simply increase the quality and quantity of comments you write. Comments rarely have bugs (though they can) and in general, we don't have enough quality comments. I am not condoning code-inflation, but the act of commenting is like a mini code review, it forces you think, refactor and improve.

Answer 6

Trying to enlighten management is a non-starter. There's an old cliché, "Are you going to believe me or your lying eyes?" What concerns your bosses are the bug counts. No amount of rationalization will tell them it's acceptable. It's more than twice as risky. In addition, you're not the only one affected by your bug count. QA has twice the work trying to identify your bugs, so management is going to want you to make less of them.

The best solution is to reduce the number of bugs you produce, period. Not only will management be happier, but you will be too. Won't you? Cause I'm pretty sure as much as you enjoy boasting you outperform your coworkers by a factor of four, you'd love to say you do it making the same number of, or even less, bugs than they do.

As you stated, "…the rate of errors made in code…tends to be impacted by the processes used when writing the code and after the code is written." If you want to alter the rate at which you produce bugs you're going to have to change the processes you use to write code.

Programmers use production methods to produce code, much as an assembly line uses methods to produce some mass-produced object. Okay, the software industry's practices are more like whittling knick-knacks from branches found in the woods. But since we're producing machines, we should employ the same rigor and discipline used for the high-speed machines of mass-production.

That includes the same techniques used in mass-production to reduce the defect rate: root-cause analysis to determine why bugs are made, and change the process so they aren't. Or at least that you catch before QA does.

1. Make up a list of your bugs. You've probably already got one thanks to the QA guys. Possibly categorized too. Type of bug, severity, the point at which the bug was injected into the code, etc.

2. Pick the largest category of bugs. Since your volume is so high, you should target that category first. Other categories include the ones easiest to find, and the ones easiest to make.

3. Knowing where those bugs are introduced into the code, look into making changes at that phase (and earlier) that prevents those bugs from happening, and ways to make catching them easier at that phase.

4. Be sure to look at non-programming related incidentals as well as they may make a difference in the quality of your work. Background music, interruptions, mealtimes, working too long without a break, hunger, etc.

What you find may lead you to go slower. On the other hand, it may help you produce even faster as you'll have less need to rework stuff you've already put behind you. As it is, four times as much code isn't close to being an order of magnitude better than your coworkers, so changing your process is most definitely the way to go.

Answer 7

Change your objective from producing the most code to helping the company move forward the most.

Since it seems you have many colleagues plus extra time available, the most effect you can have on a quicker delivery of features and bug fixes is to help your colleagues.

Help your colleagues by

• using code reviews to improve code quality and education.
• creating process automation to make their work faster and their lives easier.
• writing better tests with them
• attacking technical code to improving the code base
• being the go-to guy that is always available to help others.
• writing / improving tools that help with developer productivity
• making the case with management for better tools /equipment / working conditions for your co-workers if you have more clout.
• preparing for and leading educational sessions on writing better code.
• practising humility

Answer 8

So, how to deal with the fact that increased productivity will lead to an increased number of bugs?

Your boss is the only person who can answer this in your case. Talk to him, point out your better ratio and let him decide. If his decission doesn't make sense, it is time to look for a company with your way of thinking.

As professional you should be able to work with any given client conditions, just make sure they understand the consequences. A nice bug/stories chart can help your boss understand how much your productivity will sink if you take the time to produce less bugs.

You also need to consider these points:

• complexity of stories, for example simple getter/setter wrappers versus statistical calculations or real time programming or even real time statistical ...
• severity of bugs, is it small typos on text fields or wrong calculation results, program crashes
• cost to fix the bugs, both if you do it now, later or if someone else has to understand your code because you left

Answer 9

The situation is that you work four times as fast as the average programmer, but you make twice as many mistakes in a given amount of time. Relative to the amount of work you do, you actually make HALF as many mistakes as "average."

You might try to point out your low mistakes to work ratio, or even mistakes to completed product (which you can complete in one-fourth the normal time). Most bosses will accept that.

There are a few bosses that won't because they work with an "allowance" mindset of mistakes per time. Then you might slow down your work pace, doing TWICE as much work as average within a given time, and make the same (or fewer) mistakes because you have more time to check your work.

Answer 10

If your boss wants you to improve the quality of your work, then improve the quality of your work.

You should be aiming at zero bugs, not aiming to produce only twice as many as the next best programmer.

Answer 11

You should tell your boss that the metrics he is using is quite flawed. If you take a look at turnovers by guards in the NBA, you will find that they tend to have higher numbers than forwards. But, that is because they are handling the ball more. If a non-starting guard plays 1/5 as much as a starting guard and turns the ball over 3 times on average .vs. a starting guard that turns the ball over 7 times per game - at first glance it might look like the starting guard is worse. But, if you divide the number of turnovers by the number of minutes played, then clearly the starting guard has much better numbers based upon minutes played.

Likewise, you have much better numbers if the number of bugs is divided by the number of story points completed. So, that is what I would propose to your manager. Change the metric to be the number of bugs divided by story points completed, or at least add a new metric for this if the total number of bugs per developer is needed. But, since some story points are much harder and more complex than other story points, there can and will be quite a bit of variance and this needs to be taken into account by the manager as well.

What I don't think you should do is to slow down.

Answer 12

Measure value added

Argue that what really counts is the value you add. Then go and introduce rough (manual) measurement of that:

• Estimate the value of the functionality you produce
• Subtract your salary
• Subtract the estimated cost of your bugs (at least the cost to fix them)
• Subtract the estimated cost of all other Technical Debt you produce

The remainder is your value added. Likewise for everybody else.

These estimates are hard, but even rough ones may make the point. And the mere process of discussing these estimates is useful for the team and the project.

Answer 13

Top programmers have a tendency to write very regular code that is easy to debug and understand, they utilize available tools (static analysis, compiler errors, debug code) to the max. Also, top programmers already learned the value of unit-testing through there own hard experience.

So while initial amount of problems per line is the same, the problems are weeded out faster.