Should organizations penalize developers for the amount of defect reports filed against code they worked on? [duplicate]

Question

Should organizations penalize developers for defect reports filed against their work products?

I was having a discussion with my friend where he asks if a manager taking the number of defects filed against a developer is justified. My take is no, because developers can introduce defects, but by holding those against him can cause unnecessary negative feelings in the developer's mind.

What are the problems with penalizing developers based on defects that they inject? Does your organization penalize developers for creating defects in work products?

Answer 1

Sounds like it would do more harm than good. Ignoring for a moment whether it is fair for a manager to do that, let's look at the logistics...

Problem 1: Are all bugs created equal?
Developer 1 introduces a bug: Erases all customer data and curses at them.
Developer 2 introduces two bugs: Form labels are not left aligned, and the calendaring feature is off by 1 second if an event is created that spans two leap years.

So clearly developer 2 deserves more grief from their manager because they have double the bug rate. Of course not, so you come up with a bug rating system so developers with trivial bugs don't get dinged so hard. But wait, should the system factor in a modifier for a developer who is clearly making the same trivial mistake repeatedly and wasting the tester's time because they never learn from their mistakes? Maybe, hmmm. This is complicated.

Problem 2: What counts as a bug?
Manager - This report was supposed to include a running total, that's one bug for you!
Developer - That wasn't in the requirements, that's a FEATURE not a bug.

Problem 3: How do you group bugs?
Developer - "[Manager's Name], the testers filed 10 bugs against me because the velocities were incorrect on 10 different screens, but that was all related to a single bug in the getVelocity function. We argued for 3 hours about it, but they won't budge. We would like a sit down meeting with you to decide how many bugs should be filed. Oh and by the way, there is no way we are going to hit the code complete deadline tomorrow."

Problem 4: More SLOC probably means more bugs
Developer 1 sits on his butt all day, but manages to write 3 bug-free lines of code between arguments on Reddit over Arizona's immigration law.
Developer 2 works hard all day and churns out a fully functional AI that won't kill John Connor the first chance it gets."

So obviously you want to penalize the developer who makes more progress and/or takes more risks by innovating, right?

Summary
There are probably workable solutions to several of these, but as a manager of a programming team trying to meet a deadline do you really want to have everyone spending time arguing about what counts as a bug, what counts as a discrete bug, the importance of a bug, etc.? None of these things move your project forward and this will be poison for teams who will be forced to compete on issues that have no meaningful impact on the actual software being created. Not to mention what it does to your employee culture to focus this much effort on finding ways to make sure that every employee's mistakes are meticulously recorded so they can be thrown back in their face later.

Inevitably you will have developers cajoling testers to work around your bug tracking system and report issues directly so they can fix them without it going in their "PERMANENT FILE". Then you don't even have an accurate accounting of bugs or what people are really working on.

Then there is the issue of adverse impact. That is HR talk for, you better have pretty good documentation before you start penalizing employees, especially financially. And if any of them are a protected class (minorities, veterans, women, handicapped, etc.) you better be triple sure that whatever system you have set up doesn't discriminate against one of them based on membership in that class (or that a judge could be convinced as such), even if it is just an unintended side-effect of the plan.

So ultimately, you are not creating incentives to create less bugs, which is hard, but rather to negotiate away bugs by minimizing their importance or blaming them on someone else.

Short Version No.

Answer 2

Programmers are notorious for optimizing what managers start rewarding. If you reward LOC, then you get lots of whitespace to pad out the lines of code metric. If you try to punish by bug count, you will start getting into wars where the developers claim that X is not their bug (the bug is in the compiler or API or just somewhere else) - and the bug filed against them is wrong.

The last place I worked had a developer "in charge" of a product. While the "in charge" dev never changed over the years, others would work on the project when the seasonal load made it necessary. No one pointed fingers and said "you wrote that bug!" Instead, the goal was to reduce bugs in the products.

The places I've worked that had penalties for bugs also had very high turn over. The finger-pointing contributed to a hostile environment which led to the departure of developers.

Answer 3

Developers should hold themselves accountable for the quality of the code they produce, and they should expect that their manager will do the same. But that doesn't mean that some developer should get a demerit every time a bug is reported. No manager in their right mind would say:

Well, Johnson, you checked in 500 lines of code last week, and so far 241 bug reports have been traced to that code.

It's a lot of work to figure out who to blame for every little defect, and the number of bug reports doesn't always tell you much about the quality of the code. But a good manager should notice problems and find ways to fix them. He or she might say:

Johnson, that was one stinker of a checkin last week -- we've been getting bug reports at three times the normal rate. We need to get that fixed ASAP, so I've asked 'Eagle-Eye' Ferguson to go over the code with you. I'm sure the two of you will figure it out, and ol' Eagle-Eye might show you a few tricks along the way.

Every developer should expect the manager to notice their performance, including the general quality of their code. Developers should also expect their colleagues to notice -- it's all right there in the repository, after all. So if you're that guy with the high bug count, don't be ashamed to ask for some help. The rest of the team probably already knows where you're at, and if you make a concerted effort to improve the team (and the manager) will notice that too.

So, yes, you should be accountable for the code that you write, but no, the organization shouldn't have a policy of penalizing individual developers for specific defects, or for exceeding some number of defect reports. That's just going to encourage people to find ways to avoid the blame, like pointing fingers or being so careful that productivity drops. It doesn't help the development effort if people are afraid to accept responsibility for what they write. BTW, it also doesn't help to turn it around and reward specific bug fixes. ;-)

Answer 4

No, they should not. This is no manual labour and none creates bugs intentionally. How can we expect productivity from a sad horrified programmer? Things should be cool around him. Ultimately noone get benefitted from penalizing.

Answer 5

I say that this is a bad idea that will do little but create a hostile work environment.

I would even go so far as to say that ANY NEGATIVE REINFORCEMENT as a motivator for job performance will create an unhappy workforce and high turnover. Even good developers will feel stressed about slipping.

Negative reinforcement is even demeaning for children and dogs, why would you treat a professional in such a way?

Instead use positive reinforcements for LOW DEFECT NUMBERS and actively try to help members of your team who are struggling. If it doesn't work out with somebody over the long run then just let them go.

Answer 6

I'll buck the trend here, go out on a limb, and give a resounding YES!, but with a certain qualification: only for developers who are at least a standard deviation worse than their peers.

Let me break that down a bit, because there are a some loaded issues here. First up is the word "peers". It's certainly not right to compare the work of a 10 year veteran with that of a new hire fresh out of college. The newbie has less experience, but the pro is given more difficult assignments.

Next up is standard deviation. Programmers will historically on one hand resist any attempt at quantitative measurement of their work with claims of gaming the metrics, and on the other both practice the art of daily finding ways to quantify the unquantifyable through software and quote metrics about how the rockstar programmers are X times more productive than the average. There is some obvious truth on both sides.

My belief is that the best of use metrics, even simplistic ones like "lines of code", is for identifying stand-outs. You set a boundary of a standard deviation or two away from the norm, such that individuals programmers will be unable to break from the pack. Don't look for the top or the bottom programmer. The whole point here is that most of the time, no one is identified and so there's nothing to game. But every now and then it'll help you find that gem of a programmer, or tell you who your real under-performer is. And then you know what you do? Nothing. At least at first. Instead, think of it as unconfirmed intelligence. Use the information to watch the person, and look to confirm it elsewhere.

If you have a programmer who is checking bugs an order of magnitude more often than his peers, first find something to corroborate his overall performance and then go have a talk with him. And the first time, it probably should just be a talk.

Answer 7

My opinion is no because if an organization does penalize developers for the amount of bugs filed against them then the developer may be less productive in order to avoid the penalties assuming they are substantial. There can also be conflict over what bugs would be filed against a developer as there may be a QA person taking a liberal view of what constitutes a bug that is another factor here. While one could try to balance the penalties, I'd be curious to see how well that is done in reality.

Answer 8

My former coworker told me a story of a place where this practice was actually enforced: developers were penalized for each bug found in their code, and QA was rewarded for each bug they discovered. The result was that developers simply stopped developing new code because that was the only way to protect themselves from penalties.

Answer 9

Developers are not naturally lazy or ignorant to what's best. In my opinion, it would be more productive to give the developers a more accurate timeline, and better specifications prior to the project kicking off.

Answer 10

Definitely not.

Its like demoting doctors because some of there patients die, without considering how serious the illness/injury was.

There are some projects where Defects are just unavoidable. If you have project sponsors with conflicting needs, user requirements that conflict with security guidelines, ill though out last minute changes then there will be defects raised no matter what the quality of your code.

There is also the perennial problem (at least in most large organizations) of the infrastructure changing around you. You code for V1.25 of the database on V3.8 of the OS but another department upgrades to V2.0 and V4.0 just before you go live. (Or downgrade, in one case we coded for V5.0 J2EE only to have the upgrade cancelled and had to defactor down to V4.0 days before the release -- there were a lot of defects.).

Lastly we have the pragmatic users who raise defects instead of change requests as the bureaucratic burden is significantly lower.

Answer 11

No. It sounds trying to treat a symptom, not the root problem. Instead of focusing on how many bugs there are in a piece of code and who introduced them, why not focus on developing a system that helps eliminate bugs in the first place?

Tools like Unit Testing, Integration Testing, Test Driven Development, Static Code Analysis, Design Reviews, and Code Reviews can often catch a fair number of bugs before they get into the code base. Continuous Integration also help identify issues early rather then later.

BTW, the logical follow on to the OP question: If I inherit a piece of code filled with bugs, should I be penalized for all of the remaining bugs if I fix one? By the original proposition, since I worked on the code, I am now responsible for the bugs.

Answer 12

What is the motivation for penalizing developers for their defect rates?

Surely the goal is to try and measure the quality of the code being written by the developer.

The problem is, quality is a bit amorphous: it is really hard to measure. In fact, you can't measure it directly, you have to try and approximate it by measuring other things.

Also, picking a single KPI - such as # defects - results in gaming of the system. If you suggest penalising each developer $10 per defect, how long do you think it would be before one of the developers makes a deal with a tester ... I'll pay you $5 for each defect you tell me about that doesn't get logged.

Answer 13

Let them repare all bugs related to their code, forward them the complaints from the testing department, and they should already learn that less bugs is better for them.

And when everything goes alright, and no bug reports are coming in, a clap on the back as reward and a few kind words should do the rest.

Answer 14

Although I don't agree with the premise ...

It would could only be feasible if you rewarded the programmer that takes on more technically challenging features as well - "higher risk, higher reward". Otherwise why would a programmer ever take on challenging work in that environment, since any technical challenge is a liability (not an opportunity for reward).

I agree that it would create a very hostile environment as well. In my experience a significant amount of defects are due to requirement ambiguity or just a lack of a requirement. So I think this could also lead to those defects being explicitly assigned back to the business analyst, systems engineer, or project sponsor and they should be subject to that the same penalty - which wouldn't be a bad thing.

Answer 15

I would say counting them is a mistake. There is no way you can grind something like that down to a number. However, if a particular developer's code seems to have more problems than similar code written by others, and this is a consistent pattern, then I would say that developer is probably not as good, all other things being equal. It's a fact to be put into the mix.

In any event, I've been in this business for decades, and I can't say I've ever seen a programmer who had an unusually high or unusually low bug rate. I've seen programmers who were faster or slower, programmers who could get jobs done or who ran into trouble. I've seen programmers who naturally write very easy to understand code, very fast code, or very confusing code. But I've never seen a programmer with an atypically high or low bug rate. Maybe I just haven't been paying close enough attention.

Answer 16

I'm not opposed to this idea - but ONLY if there is consensus about the defects: a designated group of senior technicians that rules on them. Users, BA's and managers often are not qualified to decide on what is a code defect - and it's always dangerous to put something like this under the control of one (or any) individual.

But I don't think it's really necessary - when a developer is consistently writing bad code, if his seniors and peers are doing their job, it won't be a secret for long and the developer will suffer the consequences with a formal system of 'demerits'.