There are a lot of strong opinions surrounding the debate but obviously this isn’t actually a matter of opinion, it’s a matter of facts. So we should look at empirical research. And the evidence from that is clear:
Yes, static typing is worth the trade-offs — and not just by a bit, but in fact substantially. In fact, solid evidence shows that static typing can reduce the number of bugs in code by at least 15% (and this is a low estimate, the actual percentage is almost certainly larger). That is a shockingly high number: I think even most proponents of static typing wouldn’t have thought that it made such a drastic difference.
Consider this: if somebody told you that there was a simple way to reduce the bugs in your project by 15% overnight, that should be a no-brainer.1 It’s almost the proverbial silver bullet.
The evidence is presented in the paper To Type or Not to Type: Quantifying Detectable Bugs in JavaScript by Zheng Gao, Christian Bird and Earl T. Barr. I encourage everybody to read it, it’s a well-written paper that presents exemplary research.
It’s hard to summarise succinctly just how rigorously the authors performed their analysis but here’s a (very rough) outline:
TypeScript and Flow are two programming languages based on JavaScript that, while remaining otherwise compatible, add type hinting and static type checking. This permits augmenting existing code by types, and then to type check it.
The researchers collected Open Source projects written in JavaScript from GitHub, looked at resolved bug reports and attempted to reduce each of the reported bugs to a piece of code that would be caught by the static type checker of TypeScript or Flow. This allowed them to estimate a lower bound of the percentage of bugs could be fixed by using static typing.
The researchers took stringent precautions to ensure that their analysis wouldn’t consider a non-type related bug as being related to types.2
Compared to past studies, this new study has particular strengths:
- There is a direct comparison of static vs dynamic typing, with few (if any) confounding factors, since the only difference between JavaScript and TypeScript/Flow is the typing.
- They perform replication across multiple dimensions by checking both TypeScript and Flow (i.e. different type systems), and by having different people reproduce the (manual) type annotation to fix the bugs. And they perform this across a large number of code bases from different projects.
- The paper measures the direct impact of static typing on fixable bugs (rather than some more vague quality).
- The authors define a rigorous model of what to measure, and how, up-front. Furthermore, their description is incredibly clear and makes it easy to analyse for flaws (it’s always good when a research paper opens itself up to attacks: if no attacks manage to dent its arguments, it comes out even stronger).3
- They perform proper power analysis so that their sample size is sufficient, and their subsequent statistical analysis is airtight.
- They are overly conservative to exclude confounding explanations, and only measure a single moving part. Furthermore, they restrict their analysis to bugs that are immediately fixable by including types, and exclude anything that might require more advanced refactoring to accommodate typing. So in reality, the effect is plausibly a lot bigger, but certainly no smaller than what they reported.
- And finally, they don’t find a slight effect but a staggering difference. Despite their overly conservative procedure, even at the low end of the 95% confidence interval they find that there are at least 10% of bugs that would simply vanish with minimal added type checks.
Unless there is a fundamental flaw in the paper that nobody has yet discovered, the paper conclusively shows a large benefit of static typing, at almost no cost.4
On a historical note, research on typing disciplines in programming has had a rocky start because, for a long time, the evidence wasn’t clear at all. The reason for this is that doing systematic experiments to examine the effect of static vs dynamic typing isn’t easy: a systematic experiment must isolate the effect we’re investigating. And unfortunately we can’t isolate the effect of the typing discipline, since it’s tied to the programming languages.
There actually were programming languages that allowed both static and dynamic typing in different dialects (e.g. VB with Option Strict
On
or Off
, or statically typed Lisp). However, these weren’t well suited for a direct comparison, most importantly because there were no existing, sufficiently large code bases that allow direct comparison. At best we could compare them in “laboratory settings”, where test subjects randomly solve a task in the statically or dynamically typed variant of the language.
Unfortunately these artificial programming assignments don’t model real-world usage well. In particular, many of them are small in scope and solve a well-defined problem that can be summarised on half a page of text.
Luckily that’s in the past, because TypeScript, Flow and JavaScript are indeed the same languages except for static typing, and because there is an extensive real-world dataset of code and bugs to sample from.
1 Inspired by a quote from the original paper.
2 I am not entirely happy with this: one of the main strengths of statically typed languages is that ostensibly type-unrelated problems can be phrased in ways that can be statically type-checked. This transforms many logic errors into type errors, which drastically increases the rate of bugs that can be caught by static typing. In fact, the paper roughly classifies type-unrelated bugs and I contend that a large percentage of those could in fact be caught by static typing.
3 I invite anyone, especially proponents of dynamic typing, to try to find unaddressed flaws in the analysis. I don’t think there are many (if any), and I’m confident that no potential flaw would materially alter the outcome.
4 I suspect that the actual cost of static typing in real, large-scale projects is nonexistent, since it then becomes a natural part of the architecture and might even simplify planning. Fixing static type errors takes time, but much less than errors discovered later. This has been extensively empirically studied and has been known for decades (see e.g. Code Complete).
Java
orC#
would be inconclusive, their way of providing it is NOT the only one...