With TDD, tests obviously fail first.But is that really true?

Question

I have been taught that with TDD, the tests "naturally fail first but it is a good habit to run them anyway to see the red light". Well, but I am quite sure a unit test written first for a new feature might actually pass, if such a feature is already implemented (and for example somehow inhibited).

I am borrowing the following example from this answer:

    public string ShowScoreEvaluation(byte points)
    {
        switch(points)
         case 3:
            return "You are good!";
         case 2:
            return "Not bad!";
         case 1:
            return "Quite bad";
         case 0:
            return "You suck!"
      return null;

    }

    //caller code
    if (Points>0)
      Display(ShowScoreEvaluation(Points));

In the code above, the calling code does not expect to call the method when Points=0. Maybe during the implementation of that method, the programmer just put something there (as a joke or a placeholder) even for the case when points=0.

And now imagine, that you join the project and get a new request "When player has 0 points, show an encouraging message blabla". You write a unit test with Points=0 and expecting a string with length>0...and it did not fail, although you would expect it.

Could this not happen in the real life? I mean, to me this is the reason why I truly should see whether a new test fails, because then how could I know what made it pass?

Answer 1

I guess for what you really want to know, your example is not well suited, so let me describe a better one.

There are several programming problems where the most elegant, shortest implementation is not the one which just serves a few test cases, but one which solves the problem in a more general fashion than specified by the tests.

For example, one can implement a sorting algorithm incrementally by writing some test cases in a TDD fashion for sorting two, three or four elements. Maybe one starts with an implementation which can only sort two elements, like

int[] Sort(int[] list)
{
     // ... maybe some handling for lists with 0 or 1 element here ...
     if(list[0]>list[1])
        return new[]{list[1],list[0]};
     else
        return new[]{list[0],list[1]};
}

Now, a test which passes 3 elements into this function will fail. However, when you extend this code to 3, 4 or more elements, you will quickly reach the point where a general algorithm like a bubble sort or insertion sort (which works for arbitrary list lengths) is simpler than an implementation which works only for a fixed number of elements. In fact, by doing TDD, you might have started with a convoluted algorithm which compared up to 4 elements one-by-one against each other, but in the refactoring step, you replaced this by a more general sorting algorithm, which makes the overall implementation simpler.

Now it should not be very astonishing when you add further tests with more elements, these tests won't fail, though you followed all the TDD rules literally, and though you did not implement more code than needed to satisfy all existing tests.

And yes, that is a real-world case, far more likely to happen as this answer pretends. I have encountered this situation lots of times for all different kind of problems in string processing, set manipulation, mathematical or geometrical algorithms: general implementations are very often simpler than specialized ones.

So what should one do in this case? Leaving out the additional tests with 5, 8 or 20 elements, just because the code "is already complete, and the additional tests won't induce any more code changes"? I would not recommend this - it is obviously good to have such additional tests for a complex algorithm, it will give you much more confidence into the correctness of the code.

The better alternative here is to make your additional tests artificially fail for a short time. The main purpose of seeing any tests failing first before they pass in TDD is to make sure the test is actually executed - it is a "test for the test". You could, for example, add a statement like

if(list.Length>=5)
    return null;

somewhere inside the Sort function, and remove it after you saw it the test failing. That proofs your new tests are actually executed, and you did not intermix it with some existing test.

---

Since you asked for another example: lets say you have a function

`string TrimNumeric(string value)`

which is expected to replace non-numeric characters from the beginning or end of the input string value and return the result. What happens with non-numeric characters in the middle is not specified so far. The following test cases are already passing:

Assert.AreEqual("123",TrimNumeric("abc123"));
Assert.AreEqual("123",TrimNumeric("123xyz"));
Assert.AreEqual("456",TrimNumeric("abc456xyz"));

Now you get an additional requirement: non-numeric characters in the middle shall be stripped as well. You start by adding a test:

Assert.AreEqual("123",TrimNumeric("1a2b3cxyz"));

In such a case, if you don't know how TrimNumeric is implemented internally, there is no indication if this test will fail or not. Indeed, if TrimNumeric was implemented in a straightforward, simple, and general fashion, by iterating over all characters and keeping only the numeric ones, it is IMHO pretty likely this test will pass immediately. Nevertheless it should be clear why is necessary to write such a test. Maybe the test will not pass, if the implementation looks differently. But if it passes right from the start, make sure you make it fail at least temporarily, to be sure it gets executed.

Answer 2

I am quite sure a unit test written first for a new feature might actually pass

Yes, it might. There are a number of reasons why a feature might already 'be there', untested, but if you are following strict TDD, the most common reason would be that the refactor step in your red-green-refactor cycle made the behaviour of your code more general, and able to cope with cases beyond your initial specific test cases.

The other most common reason in a professional context is probably that that someone has previously worked on the code and put the feature in there without a matching test!

In either of those cases, it's still well-worth writing the test. You should only consider that your product 'has' a feature if that feature is tested and documented.

Answer 3

Your philosophy is correct.

(but your example is not a valid argument.)

The idea of a TDD test failing "naturally" is not really part of the rules. And you are adding an assumption that code written while not following the rules, will still obey your assumptions.

The rules of TDD state:

1. You are not allowed to write any production code unless it is to make a failing unit test pass.
2. You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
3. You are not allowed to write any more production code than is sufficient to pass the one failing unit test.

(emphasis mine)

So when you say this:

... a unit test written first for a new feature might actually pass, if such a feature is already implemented...

That code should have already had tests written for it. And when writing those tests, they should have "failed" before the code was written.

So why do I say your philosophy is correct?

It all depends on what test case you choose to write first.

Let's say that I am going to be create a function that prints out every character in a string passed into it. My goal is to end up with something like this (pseudo code):

function printLetters (String input) {
    for(var letter in input) {
        console.print(letter);
    }
}

Now, if I am sitting down to work in this function, my first test case might be this:

"When an empty string is passed into my function, nothing is printed".

Now, by rule 2, if we count "compile errors" as failing test, then once I try to call the function (that I haven't created yet) I have a failing test! YAY!

But, what WILL happen is that eventually I would start writing the prod code, and get to this point:

function printLetters (String input) {

}

And now my test is passing. So, I know that my code is not done, but I don't have a failing test.

But what does this mean?

This just means that we have not written the rest of the test cases to satisfy the business rules!

So I need to add more test cases (which will fail).

And, if I am following the "rules", I better make sure that the test I am writing is failing (AND FAILING FOR THE CORRECT REASON!) before I start writing the production code.

Answer 4

Could this not happen in the real life?

In an infinite universe, almost anything can happen... but if the desired behaviour already exists, it is very unlikely (not impossible) that you would receive a feature request for it.

...[you] get a new request "When player has 0 points, show an encouraging message blabla". You write a unit test with Points=0 and expecting a string with length>0...

The problem is that your suggested test does not match the feature you are trying to drive out.

The test implied by your feature would be “with Points=0, expect a string matching "an encouraging message blabla"” -- which would fail for the current implementation.

Once you have seen it fail, you would make the simplest change possible to make it pass. What would that be? It depends on what other tests are in place.

I would guess the simplest change would be:

    switch(points)
     case 3:
        return "You are good!";
     case 2:
        return "Not bad!";
     case 1:
        return "Quite bad";
     case 0:
        return "an encouraging message blabla"
  return null;

But if you only have existing tests for cases 1-3, the simplest change could be:

    switch(points)
     case 3:
        return "You are good!";
     case 2:
        return "Not bad!";
     case 1:
        return "Quite bad";
  return "an encouraging message blabla";

This, in turn, could lead to a further requirement to return something else in a default case.

Answer 5

Could this not happen in the real life?

It does to me; especially when I happen to choose a simple initial test, where the default implementation happens to return the correct value.

For instance, suppose we are trying to develop a function that adds up the numbers in a list. If we happen to choose an empty list as our first test (why not? it's easy to set up), then the implementation return 0; is going to produce the correct result.

I have been taught that with TDD, the tests "naturally fail first but it is a good habit to run them anyway to see the red light"

What I eventually came to decide is that RED-GREEN is about test calibration; the goal during that part of the process is to verify that the test you have created actually evaluates your production implementation.

RED/GREEN is the most common pattern of calibration activities, but sometimes GREEN/RED/GREEN (or, if you prefer, GREEN/RED/REVERT) happen instead.

Calibrated tests that pass naively are still valuable, in that they constrain your choices during the refactoring phase (more precisely, they help to distinguish changes that are refactorings from changes that have altered the observable behavior of your system).