Isolated Unit Tests and Fine Grained Failures

Question

One of the reasons often given to write unit tests which mock out all dependencies and are thus completely isolated is to ensure that when a bug exists, only the unit tests for that bug will fail. (Obviously, an integration tests may fail as well). That way you can readily determine where the bug is.

But I don't understand why this is a useful property. If my code were undergoing spontaneous failures, I could see why its useful to readily identify the failure point. But if I have a failing test its either because I just wrote the test or because I just modified the code under test. In either case, I already know which unit contains a bug.

What is the useful in ensuring that a test only fails due to bugs in the unit under test? I don't see how it gives me any more precision in identifying the bug than I already had.

Answer 1

But if I have a failing test its either because I just wrote the test or because I just modified the code under test. In either case, I already know which unit contains a bug.

But, "the code under test" is not only the code of the class/module being tested, but also the code executed by its dependencies.

If I have

public TaxCalculator {
  public Currency getTaxAmount(Currency barePrice) {
     ...
  }

  public Currency getFinalPrize(Currency barePrize) {
     return barePrize.add(getTaxAmount(barePrize));
  }
}

and I test getFinalPrize without mocking getTaxAmount and I get a failure, then all I know is that either there is a failure in getFinalPrize or in getTaxAmount.

Granted, for a situation this simple you would see that getTaxAmount test has also failed and maybe you will automatically link both fails. But if you think of a bug in some class that is widely used, and you suddenly get tens or hundred of test failures, stablishing that relationship would not be that easy.

Answer 2

Who gives that reason? I consider it to be a weak reason, in fact almost useless. The point of having unit tests is to ensure that no detectable defects are in the code under test. When the test suite fails, you know that there is at least one defect, so you can't publish your code base - it doesn't matter whether exactly one test case fails or more.

Clearly it would be nice to be able to read off from the title of the single failing test exactly what the problem is, but that never happens in practice. It takes an inordinate effort to arrange matters so that every test really does exercise only exactly one bit of code, and no other test does the same, and that effort is almost never worthwhile. It is much more important that every code path is exercised at least once than exactly once. And if a change breaks more than one test case at a time, well usually you investigate one of them and fix the problem, at which point the other failures disappear as well - it's rather like compiler diagnostics in that respect: only the first failure is meaningful, the others may be spurious.