The proponents of functional programming languages assert that functional programming makes it easier to reason about code. Those in favor of statically typed languages say that their compilers catch enough errors to make up for the additional complexity of type systems. But everything I read on these topics is based on rational argument, not on empirical data.

Are there any empirical studies on what effects the different categories of programming languages have on defect rates or other quality metrics?

(The answers to this question seem to indicate that there are no such studies, at least not for the dynamic vs. static debate)

  • 6
    As you can probably imagine, there are a ridiculous number of confounding factors involved. There are "empirical studies" out there, but they're little more than well-documented anecdotes and should be given about as much weight as that warrants. – C. A. McCann Aug 4 '11 at 17:39
  • 1
    possible duplicate of Dynamically vs Statically typed languages studies – Steven A. Lowe Aug 4 '11 at 17:45
  • @Steven: This question appears to be scoped more broadly (perhaps too broadly). – Robert Harvey Aug 4 '11 at 19:08
  • If you found such a study, what would you do with it? – JeffO Aug 4 '11 at 20:02
  • @Robert there are COCOMO studies along these lines, but they are meaningless - unless you've studied your own team, and that's nearly impossible to do objectively – Steven A. Lowe Aug 4 '11 at 20:34

There is some research in academia on this subject. Here are some examples I know of, although you should treat the conclusions with caution:

  • An experiment about static and dynamic type systems: doubts about the positive impact of static type systems on development time, Stefan Hanenberg. In Proc OOPSLA, 2010. ACM Link

  • An Empirical Study of Static Typing in Ruby, M. Daly, V. Sazawal, J. Foster. In Proc PLATEAU, 2010. PDF

  • A Controlled Experiment to Assess the Benefits of Procedure Argument Type Checking, Lutz Prechelt and Walter F. Tichy. IEEE TSE, 1998. IEEE Link

I'm sure there are other papers. Generally speaking, however, this area is extremely controversial for obvious reasons --- it's really hard to make an objective assessment!!


One famous study is Lutz Prechelt. An empirical comparison of seven programming languages. IEEE Computer [33(10):23-29], October 2000

Prechelt discusses program reliability, and also examines execution time and memory consumption.


Although it's not related to code quality as such, this study looks at how novices learn using different languages. In particular, they compare how novices fare when learning Perl vs Quorum, a teaching language the authors wish to compare. What's really cool about this paper is that they actually come up with a control language where the syntax is generated randomly, as a sort of "placebo" language. This approach might be really interesting if applied to languages and code quality and help control some of those tricky confounding factors when comparing languages.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.