I have been told that the average number of bugs/defects per line of code is "constant" for different programming languages. 10 KLOC of Ruby would have the same number of bugs as 10 KLOC of c++. The argument is usually used to promote the use of expressive languages (think python/ruby over c++/assembly) since the number of lines to describe the same functionality would be smaller.
Does anybody know where this claim comes from? Does higher-level languages lead to fewer bugs?
Contrary to intuition, the number of errors per 1000 lines of does seem to be relatively constant, reguardless of the specific language involved. Steve McConnell, author of Code Complete and Software Estimation: Demystifying the Black Art goes over this area in some detail.
I don't have my copies readily to hand - they're sitting on my bookshelf at work - but a quick Google found a relevant quote:
Industry Average: "about 15 - 50 errors per 1000 lines of delivered code."
(Steve) further says this is usually representative of code that has some level of structured programming behind it, but probably includes a mix of coding techniques.
Quoted from Code Complete, found here: http://mayerdan.com/ruby/2012/11/11/bugs-per-line-of-code-ratio/
If memory serves correctly, Steve goes into a thorough discussion of this, showing that the figures are constant across languages (C, C++, Java, Assembly and so on) and despite difficulties (such as defining what "line of code" means).
Most importantly he has lots of citations for his sources - he's not offering unsubstantiated opinions, but has the references to back them up.
It seems to boil down to this: The average number of defects per kloc seems to be more a property of the fact that developers are fallible humans than of the peculiar advantages or disadvantages of a particular language or platform.
(Aside: If you don't already have Code Complete, go buy yourself a copy and read it thoroughly - it's well worth the investment. )
Update: There's another factor at play with some of the answers here - large scale statistics are useful for making general predictions but not specific ones. Consider, population mortality tables can predict about how many people will be killed in traffic accidents this year but can't tell you which people will die. Similarly, industry statistics that show a relatively constant number of defects per kloc can't be used to predict how well - or how poorly - a particular developer will perform or what will happen on a given project.
The claim is - at best - naive.
SLOC isn't exactly a reliable metric for anything useful, except perhaps comparing the size of two or more projects. Furthermore there are two distinct types of SLOC, physical LOC and logical LOC, and those might differ significantly. Consider this example, from Wikipedia:
for (i = 0; i < 100; i += 1) printf("hello");
Here we have one physical LOC, but two logical ones (for and printf statements). But we could of course write the example as:
for (i = 0; i < 100; i += 1)
printf("hello");
Which would give us two physical and two logical LOCs. I think it's clear that any "bug per loc" measurement that would depend on physical LOCs would be tainted by programming style, thus our measurement would be largely useless.
If, on the other hand, we went with logical LOCs then our measurement would heavily depend on the language's syntactic idiosyncrasies. Although the resulting metric might be a bit useful when comparing projects written in the same language, it would be fairly useless for projects written in different languages.
One possible source for the claim is Les Hatton's Software failures-follies and fallacies:
We can conclude that programming language choice is at best weakly related to reliability.
Later on, the paper mentions similar defect densities for C and C++:
In a recent study comparing two similar systems of similar size, (around 50,000 lines each), one in C and one in object-designed C++, the resulting defect densities were shown to be around the same at 2.4 and 2.9 per 1000 lines respectively.
This, however, doesn't mean that "bug per LOC" is constant across programming languages, or that it would be significant if it was.
This observation is very old, and comes from a very venerable source, namely Fred Brooks in his book "The Mythical Man Month". He was a top manager at IBM, and managed many programming projects including the milions-of-lines operating system OS/360. In fact he reported that the number of bugs in a program is not proportional to the length of code, but quadratic! According to his research, the number of bugs was proportional to the length of the program to the power 1.5. In other words, a program that is ten times longer has 30 times more bugs. And he reported that this held over all programming languages, and levels of programming languages.
I don't find Bugs per LOC to be constant at all for a given language. Bugs per LOC seem like a metric some Managers use to determine the quality of developers when it comes to review time.
Now outside of that, some languages are more prone to errors or defects than others. Usually, but not always this is a lower level language than a higher one. For example coding in C versus C# (or Java.) I say usually because the reality of it and the crux of the answer you are looking for comes down to quality of developer and the coding practices in place. I have seen very good C developers with much higher code quality and lower defect counts than average Java/C# developers. This is one item that separates a senior developer from a junior one. Not how many LOC they write in a given time frame, but quality of the code the write regardless of language, LOC or time frame.
The only thing answer I can give that might relate is that the more LOC there are the more likely-hood of there being a defect and the more defects that exists.
Bugs / LOC is only relative to an individual. For businesses that implement bug tracking tools that link with their source code repository. It's possible for a manager to organize issues by developer, sorted by past issues and code changes.
A senior software developer, who is highly experienced, highly skilled, very smart and able to take on independent jobs is far more likely to have way more bugs logged in a tracking system, then a junior developer with little experience.
How is that possible?
Senior developers are often engaged in higher risk development tasks. Refactoring of code and building new systems as an example. Junior developers are often assigned to fix known issues that aren't worth the time of a senior developer.
Therefore, by task assignment a junior isn't introducing bugs but fixing them, and a senior developer is allowed the risk of introducing them, because the benefit of what they are trying to archive is more important then the minor issues that are raised completing those tasks.
The argument that a language introduces less bugs, because it can achieve more in fewer lines of code is a complete myth. Highly structured languages like C++/C#/Java force the developer to clearly express in writing what the desired instruction should be, where as languages like Python/PHP are very unstructured. Those languages allow for written expressions that not only will confuse a developer, but also the language parser.
How many bugs in Python/PHP have made it out into production servers, because there was no compiler to warn the developer that something was incorrect. When you measure bugs per LOC is that before or after a compiler has processed the source code?
Update 2019:
Compilers make no difference on the nature or number of bugs. Bugs are purely relative to the person who wrote the source code, and bugs themselves can be very subjective in nature.
FWIW, in my experience
There are two kinds of bugs: a) where the program does not meet expectations, and b) where the program cannot meet any reasonable expectations, because it crashes/hangs/won't compile.
Regardless of language, bugs of type (b) are caused by redundancy in data/class structure, where changing something in one part of the data structure puts the structure in an inconsistent/broken state until one or more corresponding changes are made in other parts. Contributing to this is redundancy of source code, where an edit to one line of code makes the code incorrect until one or more changes are made in other parts. These two types of redundancy are closely related, of course, and since programmers are not super-persons they get distracted, forget things, and make mistakes, thereby putting in bugs.
These things (again, in my experience) are not really a function of the language, but of the skill/maturity of the programmer. Programs that are much less bug-prone also tend to be much smaller, in terms of LOC, for a given set of functionality.
I've seen systems where some people write programs, while others write directories, and the former tend to "just work" compared to the latter.
I would expect that a key factor in coding errors relates to what I call the "semantic gap" between a particular type of solution definition and the code to solve it - where these are close reformulation errors would be more apparent, where the code is very different, many errors could be expected. The paradigm of certain languages closely match certain problem domains - spreadsheets are very appropriate to everyday business calculations, resulting in both very little "code" and the "code" being very close to the problem domain. The expected code is both very concise (little KLOC) and few errors. Conversely using assembler would require many KLOC and is likely to produce immense number of errors.
Instead of talking about lines of code - which are indeed a useless metric - I'd like to address this part of your question:
Does higher-level languages lead to fewer bugs?
This is different than bugs/LOC, because higher-level languages do more with less code. Implementing some feature requirement might take 500 lines of LISP vs 15000 lines of x86 assembly.
So, even if bugs/LOC is constant between all languages, the higher-level language will still yield fewer bugs.
{1≥⍴⍵:⍵⋄e←⍵[?⍴⍵]⋄ (∇(⍵<e)/⍵) , ((⍵=e)/⍵) , ∇(⍵>e)/⍵}is as likely to contain an error asint pivot = arr.Count / 2;?