He argued that tests were constantly in his way when he tried to implement a new feature. The tests would fail after he had changed the code. So he had to adapt the tests which of course increased his workload.
That is the number one way in which tests help implement new features.
The tests would fail after he had changed the code.
The code sucked. Possibly the test code sucked. More likely his code sucked. Either way, the code sucked.
Let's say that there's about a 95% chance that it was his code that sucked, 2% chance that the test code sucks on purpose ("we have to disallow case A because while it's generally perfectly fine it causes problems with strange legacy client-setup B so we're testing to make sure nobody hits it") and 3% chance that the test code sucks.
Now that's being very generous, but even if the rates were up to 99.99% this wouldn't say anything bad about that person's skill or craftsmanship. (Consider how often you hit a compiler error; that's code that sucked too bad to even run the tests, but your overall work that day may have been fantastic, as may the final product when you fixed the source of those compiler errors). Indeed, its not really generous to hit any percentages here, test code tends to be the code to blame less often just because it's changed less often so it has fewer chances to acquire new bugs. And remember we're talking about the case of an identified failure, not about how often these failures happen.
Anyway, with those percentages there is a 98% chance that something is incorrect that needs to be fixed, a 2% chance that something is incorrect outside of your control that needs to be catered for, and a 100% chance that you need to change something.
The 3% chance that the test is incorrect further break down into a few possibilities:
- The test is buggy.
- The test is no longer relevant.
- The test is invalid for a particular case and is being applied generally.
As such, even when the test is to blame, it may not necessarily be "wrong" so much as out of date, some of the time.
In those 3% of cases, dealing with tests has cost you extra work that you wouldn't have had if you didn't use tests. But once that work is done it has added assistance to the 97% of the time when something else is to blame.
And in the 97% of times the test has led you to do more work right then but the result is that the non-test code doesn't suck any more (the 95% of the time) or deals with the external out-of-your-control issue fine (the 2% of the time). Note that the 2% not-your-fault case is still perceived by the user as "this software sucks".
So, 97% (realistically, more than that) of the time the test has made the code suck less.
Let's consider the alternatives for that 97% without the unit tests finding the flaws:
- The code's problems were still there, but nothing ever hit the relevant code path in real life. (Hey, it happens).
- The code's problems were found when other code wouldn't work with it.
- The code's problems were found when another developer had to do something with it.
- The code's problems were found in feature testing or application testing.
- The code's problems were found in user-acceptance testing.
- The code's problems were found after the product had been released, by annoyed users.
- The code's problems were found after the product had been released, by delighted crackers who then made use of it to compromise your or your customers' security.
- The code's problems weren't found, but did cause incorrect financial transactions, injury, loss of life, or continuance of enemy life. (Those last few are only applicable to increasingly specialised applications and may not apply to you, but they certainly do apply to some programmers).
Any professional programmer, and any amateur who releases their work, for that matter, will have experienced the first 6 of those cases happening. Some of us have experienced the final 2. They are the scenarios we most try to avoid, but they happen.
When items 2 through 5 happen, the impact on our workload is much greater than that of a failed unit test. Very often more time is spent dealing with these issues than developing. A great many projects have never gotten out of alpha because they never got past such issues.
For the later items in the list, the impact on workload may not be the worse thing about them.
The vast, vast majority of the time you have a failed unit test, you have been saved from one of the above scenarios. Unit testing won't save you from all of them, but it will save you from many of them.
Tests that pass are nice. They can give you a degree of confidence that at least certain bugs are not present in certain units, and that's great.
Tests that fail are the tests that have earned their keep.