"A good programmer can be as 10X times more productive than a mediocre one" [closed]

Question

I had read an interview with a great programmer (it is not in English) and in it he said that "a great programmer can be as 10 times as good as a mediocre one" giving reason for why good programmers are very well paid and why programming companies give many facilities for their employees. The idea was that there is a very large demand for good programmers, because of the above reason and that's why companies pay very much to bring them.

Do you agree with this statement? Do you know any objective facts that could support it?

Edit: The question has nothing to do with experience; if you talk about one great programmer with 1 year experience then s/he should be 10 times more productive than a mediocre programmer with 1 year experience. I agree that from certain experience years onwards, things start to dissipate but that's not the purpose of the question.

Answer 1

A pretty thorough overview and analysis of research about productivity differences is provided in two articles written by Steve McConnell:

• Productivity Variations Among Software Developers and Teams: The Origin of "10x"

• Origins of 10X – How Valid is the Underlying Research?

First article (Productivity variations...) states:

...The original study that found huge variations in individual programming productivity was conducted in the late 1960s by Sackman, Erikson, and Grant (1968). They studied professional programmers with an average of 7 years’ experience and found that the ratio of initial coding time between the best and worst programmers was about 20 to 1; the ratio of debugging times over 25 to 1; of program size 5 to 1; and of program execution speed about 10 to 1. They found no relationship between a programmer’s amount of experience and code quality or productivity.

Detailed examination of Sackman, Erickson, and Grant's findings shows some flaws in their methodology... However, even after accounting for the flaws, their data still shows more than a 10-fold difference between the best programmers and the worst.

In years since the original study, the general finding that "There are order-of-magnitude differences among programmers" has been confirmed by many other studies of professional programmers (Curtis 1981, Mills 1983, DeMarco and Lister 1985, Curtis et al. 1986, Card 1987, Boehm and Papaccio 1988, Valett and McGarry 1989, Boehm et al 2000)...

^{This article also has an interesting side note:}

^{This degree of variation isn't unique to software. A study by Norm Augustine found that in a variety of professions--writing, football, invention, police work, and other occupations--the top 20 percent of the people produced about 50 percent of the output, whether the output is touchdowns, patents, solved cases, or software (Augustine 1979).}

---

Second article (...How Valid is the Underlying Research?) has been written mainly to address critical review of the first one by Laurent Bossavit:

• original critique in French: Folklore ou fait scientifique, comment les différencier
• English translation: Fact and folklore in software engineering.

In second article, in section A Deeper Dive Into the Research Supporting “10x” McConnell re-checks in more details the references used in the first article and concludes:

...As I reviewed these citations once again in writing this article, I concluded again that they support the general finding that there are 10x productivity differences among programmers. The studies have collectively involved hundreds of professional programmers across a spectrum of programming activities.

...the body of research that supports the 10x claim is as solid as any research that’s been done in software engineering. Studies that support the 10x claim are singularly not subject to the methodological limitation described in Figure 1, because they are studying individual variability itself (i.e., only the left side of the figure). Bossavit does not cite even one study — flawed or otherwise — that counters the 10x claim, and I haven’t seen any such studies either. The fact that no studies have produced findings that contradict the 10x claim provides even more confidence in the 10x claim. When I consider the number of studies that have been done, in aggregate I find the research to be not only suggestive, but conclusive—which is rare in software engineering research.

---

For the sake of completeness, list of references used in the Productivity variations... is also quoted below:

References

Augustine, N. R. 1979. "Augustine’s Laws and Major System Development Programs." Defense Systems Management Review: 50-76.

Boehm, Barry W., and Philip N. Papaccio. 1988. "Understanding and Controlling Software Costs." IEEE Transactions on Software Engineering SE-14, no. 10 (October): 1462-77.

Boehm, Barry, et al, 2000. Software Cost Estimation with Cocomo II, Boston, Mass.: Addison Wesley, 2000.

Boehm, Barry W., T. E. Gray, and T. Seewaldt. 1984. "Prototyping Versus Specifying: A Multiproject Experiment." IEEE Transactions on Software Engineering SE-10, no. 3 (May): 290-303. Also in Jones 1986b.

Card, David N. 1987. "A Software Technology Evaluation Program." Information and Software Technology 29, no. 6 (July/August): 291-300.

Curtis, Bill. 1981. "Substantiating Programmer Variability." Proceedings of the IEEE 69, no. 7: 846.

Curtis, Bill, et al. 1986. "Software Psychology: The Need for an Interdisciplinary Program." Proceedings of the IEEE 74, no. 8: 1092-1106.

DeMarco, Tom, and Timothy Lister. 1985. "Programmer Performance and the Effects of the Workplace." Proceedings of the 8th International Conference on Software Engineering. Washington, D.C.: IEEE Computer Society Press, 268-72.

DeMarco, Tom and Timothy Lister, 1999. Peopleware: Productive Projects and Teams, 2d Ed. New York: Dorset House, 1999.

Mills, Harlan D. 1983. Software Productivity. Boston, Mass.: Little, Brown.

Sackman, H., W.J. Erikson, and E. E. Grant. 1968. "Exploratory Experimental Studies Comparing Online and Offline Programming Performance." Communications of the ACM 11, no. 1 (January): 3-11.

Valett, J., and F. E. McGarry. 1989. "A Summary of Software Measurement Experiences in the Software Engineering Laboratory." Journal of Systems and Software 9, no. 2 (February): 137-48.

Weinberg, Gerald M., and Edward L. Schulman. 1974. "Goals and Performance in Computer Programming." Human Factors 16, no. 1 (February): 70-77.

Answer 2

A genuinely terrible programmer can have sub-zero productivity (the bugs they introduce take longer to fix than it would take to just do all of their work for them).

And a genuinely great programmer can do things that poor and average programmers would simply never achieve, regardless of how much time you gave them.

So for these reasons, it's hard to talk about "10x as productive" or "100x as productive".

The thing to remember, though, is that most employers of programmers do not have any need for them to do the difficult tasks that average programmers could not manage. Most code being written is websites, line of business apps, intranet apps, etc., much of it really not that difficult. The productive programmer in that environment is the one who is best at understanding and implementing the users' needs, not the one who can write the cleverest code.

Indeed, most employers of programmers would be better off with a good programmer rather than a great one, because the great one will just get bored and leave. Gotta find a good match between programmers and jobs.

Answer 3

Facts and Fallacies of Software Engineering states (Fact 2, available in amazon preview):

The best programmers are up to 28 times better than the worst programmers, according to "individual differences" research. Given that their pay is never commensurate, they are the biggest bargains in the software field.

(look sources list there for research)

Of course, if you compare the productivity of a non-programmer (or a very bad programmer) with the good one (in terms of experience and knowledge), the difference can be infinitely large (n/0 == infinity for any positive n), but it is neither fair nor sensible comparison.

Your salary may depend on multiple factors (in random order):

• Technologies used
• Country/state you work in
• Wealth of the company
• Company's business type
• Number of developers in the company
• How long do you work in the company
• Office politics

together with your personal...

• productivity
• knowledge and experience
• involvement in company's business (for startups)
• negotiating skills
• sexual attractiveness and skills, lol (well, intelligence is sexy)

Answer 4

My answer is "yes, but be careful how you use that metric".

A programmer who is, shall we say, functioning optimally, is one who creates for functionality and causes fewer errors that need fixing than his lower performing bretheren. I would not find it all hard to believe that these folks can perform at 10X the producitivity of others, particularly when you consider that a single good or bad choice made in an hour can readily have 10 hours of impact, and programmers make many such choices most days.

But...

You have be careful in you measure this. I really don't trust most measurements on productivity since I've seen endless cases where just about every known metric fails to consider something I consider vital for team productivity. So I generally hate such hard numbers for "productivity". Here's some examples:

• Lines of code (LOC) - a generally hated metric, since a thoughtless programmer can generate many horrible, verbose, inefficient, hard to debug lines of code while a good programmer creates a few, elegant, easy to fix, rarely broken lines of code in more time, but which are overall a better choice.
• Bugs generated and/or time to fix - everyone will generate some bugs, and often the most expensive bugs are generated by a series of bad decisions for which the generator of the issue is merely the last in the domino effect. Also, your great debuggers are not always your great designers - you need both.
• By almost any metric, there are great developers who are such a pain to have on a team, that 1 "super productive" developer will drive away 10 basically good developers and I rarely see someone who can do everything well, so we'll need more than 1 person on the project.
• There's no way to easily account for those wonderful people who see the problems coming before they serious and head them off, particularly when the problem is a gap in a process - faulty CM, inefficient build, a gap in testing, a security flaw - these often look like a big waste of time until you realize that they save you from disaster - there's no way to measure that.
• Also, there are folks I consider necessary on a team of a certain size that I would call "cohesion builders" - rarely the absolute top of productivity, they are usually still in the upper 20% and they do the invaluable job of helping ramp up new people, connecting the dots and making sure that the right questions get asked and the right people get kept in the loop, they write (unasked!) the key piece of documentation that everyone refers to because it is not only the right document, but it's put together just the right way. If you want 10 people to work at peak efficiency you absolutely need one of these folks and you won't get it if you put 10 super developers in a room.

Many measurement systems have tried to take these factors into account, but I have yet to see that there's one that takes all of these issues into account, so I'm never overly impressed with factors like "a good developer is 10X more productive than a mediocre one" because I have to wonder if the metric really accounts for all the work that needs to go into a successful ongoing product or a successful, thriving team.

So my big caveat is - what are you going to do with this metric? I'll use something like this to be aware that the right tools and talent can cause a big difference in how work gets done but if you try to optimize to a team where each individual produces 10X the "typical" output, you are bound for a case of frustration. Better is to find a way to get your team to do 2-3X what they were doing before by working together better.

Answer 5

In his book The Leprechauns of Software Engineering, Laurent Bossavit describes researching the 10x productivity claim. He discovered that there aren't sound numbers behind it - the claim has gone from speculation to "established fact" by a telephone game of successively more concrete claims in citation. The blog post that comprises the chapter on the 10x claim, and includes the relevant citations and misquotes, is fact and folklore in software engineering.

What he found is something like this: someone in 1968 did a study comparing people solving a particular debugging problem, and found that some of them did it 10x faster than others. From this we could conclude that some people are 10x better at solving that problem, or we could conclude that some people got lucky, or a wide variety of different things. Some people chose to quote this as (these are all paraphrases) "a study (Sackman et al, 1968) found some programmers work 10x faster than others". Then it became "studies have shown that good programmers are 10x better than average" then finally "it is common knowledge that programmer productivity varies by 10x between individuals". Then someone collects all these citations, misquoting one original source to say "many researchers believe…".

Of course, it wouldn't be a telephone game if only the veracity of the assertion changed: the multiplier goes to 11 and beyond too.

Answer 6

"The productive programmer in that environment is the one who is best at understanding and implementing the users' needs, not the one who can write the cleverest code." (From Carson63000 answer)

That key point coupled with bethlakshmi's points makes a huge point. A great developer can be great in his/her one slice of reality but break apart soon as the world changes. Being able to keep up with the business' needs is far more important than anything else. At the end of the day, unless your business is technology, the business doesn't care about technology; they need solutions. So being great with design patterns doesn't mean diddly squat to end-users who just need a data dump to show on a webpage. I've seen mediocre developers secure themselves a job by catering to the business that supports them while great developers get bored and walk away in search of a never-ending challenge. Depending on your organization and project(s), it's possible to feed these challenge-starved developers but likely, there will be a point in time when you just don't need that amount of processing power. These developers don't like to just sit idle like a processor. They'll shutdown and reboot elsewhere.

Lastly, I'll say it's OK to know who your "key" performers are, but a development "team" is still a team. To reiterate bethlakshmi, "what are you going to do with this metric?" If you need a team that behaves like a team, I wouldn't focus on metrics as these. I would realize that even the smallest player is still an important part of the team. Even at 60% less of the productivity of your key player, that one player may be giving your team something it needs. Find out what it is and try to multiply it. Don't burn out your key player by assuming he should lead the team, find ways to multiply his output as well, by contaminating the other players with that greatness. This requires a bit of creativity, not just numbers. In the end, you may learn that what makes a good programmer is not even that programmer, it could be his peers, his opportunities in the workplace or it could even be you.