In TDD, if I write a test case that passes without modifying production code, what does that mean?

Question

These are Robert C. Martin's rules for TDD:

• You are not allowed to write any production code unless it is to make a failing unit test pass.
• You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
• You are not allowed to write any more production code than is sufficient to pass the one failing unit test.

When I write a test that seems worthwhile but passes without changing production code:

1. Does that mean I did something wrong?
2. Should I avoid writing such tests in the future if it can be helped?
3. Should I leave that test there or remove it?

---

Note: I was trying to ask this question here: Can I start with a passing unit test? But I wasn't able to articulate the question well enough until now.

Answer 1

It says you can't write production code unless it's to get a failing unit test to pass, not that you can't write a test that passes from the get-go. The intent of the rule is to say "If you need to edit production code, make sure that you write or change a test for it first."

Sometimes we write tests to prove a theory. The test passes and that disproves our theory. We don't then remove the test. However, we might (knowing that we have the backing of source control) break production code, to make sure that we understand why it passed when we didn't expect it to.

If it turns out to be a valid and correct test, and it isn't duplicating an existing test, leave it there.

Answer 2

It means that either:

1. You wrote the production code that fulfills the feature you want without writing the test first (a violation of "religious TDD"), or
2. The feature that you need happens to be already fulfilled by the production code, and you're just writing another unit test to cover that feature.

The latter situation is more common than you might think. As a completely specious and trivial (but still illustrative) example, let's say that you wrote the following unit test (pseudocode, because I'm lazy):

public void TestAddMethod()
{
    Assert.IsTrue(Add(2,3) == 5);
}

Because all you really need is the result of 2 and 3 added together.

Your implementing method would be:

public int add(int x, int y)
{
    return x + y;
}

But let's say I now need to add 4 and 6 together:

public void TestAddMethod2()
{
    Assert.IsTrue(Add(4,6) == 10);
}

I don't need to rewrite my method, because it already covers the second case.

Now let's say that I found out that my Add function really needs to return a number that has some ceiling, let's say 100. I can write a new method that tests this:

public void TestAddMethod3()
{
    Assert.IsTrue(Add(100,100) == 100);
}

And this test will now fail. I must now rewrite my function

public int add(int x, int y)
{
    var a = x + y;
    return a > 100 ? 100 : a;
}

to make it pass.

Common sense dictates that if

public void TestAddMethod2()
{
    Assert.IsTrue(Add(4,6) == 10);
}

passes, you don't deliberately make your method fail just so that you can have a failing test so that you can write new code to make that test pass.

Answer 3

Your test pass but you aren't wrong. I think, it happened because the production code is not TDD from the beginning.

Let's suppose canonical(?) TDD. There is no production code but a few test cases (that is of course always fail). We add production code to pass. Then stop here to add more fail test case. Again add production code to pass.

In other words, your test could be a kind of functionality test not a simple TDD unit test. Those are always valuable asset for the product quality.

I personally don't like such totalitarian, inhuman rules ;(

Answer 4

Actually the same issue came up on a dojo last night.

I did a quick research on it. This is what I came up with:

Basically it is not forbidden explicitly by the TDD rules. Maybe some additional tests are needed to prove that a function works correctly for a generalized input. In this case the TDD practice is left aside just for a little while. Note that leaving TDD practice shortly is not necessarily breaking TDD rules as long as there is no production code added in the meantime.

Additional tests may be written as long as they are not redundant. A good practice would be to do equivalence class partitioning testing. That means that the edge cases and at least one inner case for every equivalence class is tested.

One problem that could occur with this approach, though, is that if the tests pass from the beginning it cannot be assured that there are no false positives. Meaning that there could be tests that pass because the tests are not implemented correctly and not because the production code is working correctly. To prevent this the production code should be changed slightly to break the test. If this makes the test fail the test is most likely correctly implemented and the production code can be changed back to make the test pass again.

If you just want to practice strict TDD you might not write any additional tests that pass from the beginning. On the other hand in an enterprise development environment one actually should leave the TDD practice if additional tests seem usefull.

Answer 5

A test that passes without modifying production code isn't inherently bad, and is often necessary to describe an additional requirement or boundary case. As long as your test "seems worthwhile", as you say yours does, keep it.

Where you get into trouble is when you write an already-passing test as a replacement for actually understanding the problem space.

We can imagine at two extremes: one programmer who writes a large number of tests "just in case" one catches a bug; and a second programmer who carefully analyzes the problem space before writing a minimal number of tests. Let's say both are trying to implement an absolute value function.

The first programmer writes:

assert abs(-88888) == 88888
assert abs(-12345) == 12345
assert abs(-5000) == 5000
assert abs(-32) == 32
assert abs(46) == 46
assert abs(50) == 50
assert abs(5001) == 5001
assert abs(999999) == 999999
...

The second programmer writes:

assert abs(-1) == 1
assert abs(0) == 0
assert abs(1) == 1

The first programmer's implementation might result in:

def abs(n):
    if n < 0:
        return -n
    elif n > 0:
        return n

The second programmer's implementation might result in:

def abs(n):
    if n < 0:
        return -n
    else:
        return n

All tests pass, but the first programmer has not only written several redundant tests (needlessly slowing down their development cycle), but has also failed to test a boundary case (abs(0)).

If you find yourself writing tests that pass without modifying production code, ask yourself whether your tests are really adding value or whether you need to spend more time understanding the problem space.