Am I doing TDD wrong when multiple tests can fail for the same reason?

Question

Assume a method with the following declaration

public IList<Location> GetLocations() { ... }

I write (at least) the following tests

1. Check to see if the method returns an empty collection if no locations are found.
2. Check to see if the method returns the correct number of locations.
3. Check to see if the data in the locations the method return is correct.

However, all the tests above will fail if the method returns null. I feel like a single error should only result in one unit test failure, but at the same time I do not see how to avoid it in a scenario like this. Is this just the way it works; sometimes multiple tests will fail for the same reason?

Answer 1

Your assumption is wrong,

I feel like a single error should only result in one unit test failure

every unit-test test only one aspect of the method so:

1. Check to see if the method returns an empty collection if no locations are found.
2. Check to see if the method returns the correct number of locations.
3. Check to see if the data in the locations the method return is correct.

in every above test if the method return null, test fail is the auspicable response

sometimes multiple tests will fail for the same reason?

The only acceptable answer should be Yes

Answer 2

I disagree with Infer-On's answer. Indeed, though not always achievable in practice, one should strive for having a single error cause just one test to fail.

The main fallacy of your assumptions is in your list of behaviors. There is one crucial test missing from your list based on the information you provided:

0. Check to see if the method returns null when circumstance x occurs.

This is the one and only test where your method should be allowed to return null. If it returns null in the other 3 tests you have listed, then those tests are not testing what they claim.

First, you need to make sure your behaviors (tests) are correctly stating what they want. Roy Osherove in The Art of Unit Testing goes into this in copious detail but to be very brief, he purports a test name should be of the form <method> <scenario> <behavior>. And, in fact, your first test abides by this, just in a slightly different order, which is perfectly fine (and the way I tend to do it, as it happens). Here I have separated out the components in your test name:

1. <method> <returns an empty collection> <if no locations are found>

But your second and third tests are missing the scenario.

2. <method> <returns the correct number of locations> <when what??>
3. <method> <returns correct location data> <when what??>

Let's say that the scenario (the "when what??") for both (2) and (3) is something like "when given proper coordinates". (Replace that with whatever fits in your context.)

Getting back to the main point, though, for all 3 tests, if the method is returning null then it is not given the opportunity to look for locations and find none (1) and it is not given proper coordinates (2) and (3). So those tests are failing but for the wrong reasons.

So to put into practice: The extra test I introduced above (0) handles the case of the method returning null, so the method must not return null in the remaining tests. You must instrument them appropriately with test doubles (typically mocks and/or stubs) so you can eliminate outside influences and isolate precisely what you want to test.