Software Engineering Stack Exchange is a question and answer site for professionals, academics, and students working within the systems development life cycle. It only takes a minute to sign up.
Sign up to join this community
As part of a review of all job descriptions, my company has decided to include the following as a key output:
website development completed on time, within specification and error free
Given that specifications regularly change, there is no formal change control process and environments are, shall we say, a little unpredictable, how realistic and reasonable is this KPI?
"Error Free" is far too subjective. One man's "Unfufilled feature request" is another man's "Error". Something like "Should substantially meet design specs" would be more appropriate. I've never actually seen what you describe in a job description. I've seen it for contract work, but not for employees.
I'll take an opposing position to most answers and say it's absolutely reasonable and realistic.
Will all development be completed on time? Of course it won't, not always.
Will all development be completed within specification? You'd like to hope so, but sometimes that will simply not be possible and you'll have to flag a deviation from an impossible or self-contradictory spec.
And will all development be error free? Never.
But that's what a KPI is for. It's something that can be measured and by which you can track performance and progress.
If specifications regularly change, there is no formal change control process and environments are unpredictable, then it will be a challenge to keep this figure close to "error free". But that challenge is your job, and it's a job which you will hopefully do quite well - and even better next year, as you get more practice at managing your company's own particular flavour of chaos.
Counter question: what KPIs would you propose for a programmer? It's a tough one. A lot of what we do is hard to measure.
If it is a job description, then I wouldn't worry too much about it since working towards error free code is part of a typical programmer's job (even if we can never achieve it).
However, as a KPI it is too far reaching, but don't blame the person who suggested it if they aren't programmers. Just explain that that statement sets a goal that might be undesirable for the organization. That is, "error free" is an extremely high standard for software that would cost a fortune to actually deliver. Explain that a well run software project requires decisions to be made about whether each defect is worth spending valuable developer time on.
Here's an example that makes the point nicely.
A programmer discovers that our software has a "year 3000" bug and will cease to function after Dec 31,2999. It will take 6-8 months to fix the problem. Based on the KPI is encouraged to take on this project despite having no real value to the company.
Okay, so that example is a little extreme, but in any software project there will be literally dozens of little defects discovered that similarly don't generate the ROI required to fix them. If the KPI was instead intended to imply that the programmer never introduce the defect in the first place, does it seem reasonable for ANY employee to be held to the standard of never making a mistake in the performance of their job?
Not only is it not appropriate, it is ludicrous
Testing can only prove the existence of errors, not their absence, so every program written under this engagement would have to include a rigorous proof of correctness...and 100% test coverage
"Beware of bugs in the above code; I have only proved it correct, not tried it." -- D. Knuth
Of course it is every programmer's job, and responsibility, to write code that is error free. That's a perfectly reasonable expectation. How can you be a professional programmer if you release code that doesn't work? How can you consider yourself to be a professional programmer if you release code that you don't know works?
If you hire a painter you expect him to do his job well. You expect the result of his work to be error free. If there are errors, you expect him to take responsibility for those errors and fix them free of charge. What's more, if the errors cost you money, you expect him to reimburse you. Why do you have these expectations? Because the painter is a professional.
Programmers love to blame everyone else for their errors. "My program has bugs because of the requirements, or because of the schedule, or because the Moon is in the 8th house" But there's really no one else to blame. If your program has errors, you put them there.
Our profession will never be a profession until programmers realize that the buck stops with them. That they are responsible for the quality of their programs.
Do you know why companies created Software QA departments? Because programmers weren't doing their jobs! Programmers were releasing so much crap that companies had to form whole new departments to check up on them.
How long is the bug list? It is professional to have thousands of bugs in the bug database? Quite clearly it's not. It's a reflection of bad behavior, poor discipline, and, frankly, dishonor.
We will never be a profession until we realize that is it our job to make sure that QA finds nothing.
Sadly this just sounds like a way for them to "cover all the bases", and is clearly not recommended and is likely to just engender disillusionment in the developers.
However having said that, this really only matters once you see what they do with that text during the review period. So don't overreact too quickly - there might still be sanity at the end of the tunnel.
"Error-free" as in "perfect?" As in "written by God and the angels, not humans?" (we're speaking here of program-logic and maybe hardware-logic errors)
I can't say truthfully about even one single line of code that it is without error. That's because us humans, well, we can't prove no negative hypotheses!
The best I can say is that the probability of an error is a number between 0 and 1. I reach that number by way of well- or ill-defined and well- or ill-understood software development and testing principles; by a count of the source software lines in question; by an understanding of how well or poorly I candidate, poor mutt, applies those principles in producing those lines of code; and more.
And I can express that understanding only as a probability. So the term "logic-error-free" means close to nothing.
If I saw an ad for a software engineer who produced "error-free" code I'd either apply right away or I'd run right away: the company hasn't thought much about how it develops, tests, and delivers its software. So it'll be either a great opportunity or an endless nightmare.
Of any software, though, I can easily -- and must -- say I expect code that has no errors that fall outside that sucky, murky, logic-ey stuff: code that compiles and links without errors or warnings; that is "valid html" or "valid css"; JavaScript (say) that generates no unexplained error messages or browser faults. That part I can measure straightforwardly and mark in black and white on a graph.
That part's easy as pie. Anybody can do that.
Hey, good luck in your search :-)
Am I being stupid, or does "error" not mean "fatal compiler message amounting to non-compilable code"?
By that definition, it's a very reasonable requirement...
