8

I'm trying to practice TDD in my personal project and I wonder how to deal with the situation when after adding new test it pass from the start based on existing implementation?

On the one hand the new can test provide additional documentation of the design and a protection from accidental violation of assumptions.

On the other hand if test pass without any code change then it is "suspect" whether or not it really tests what it should.

Basically what can be done to confirm correctness of the test that asserts already implemented behavior?

  • 5
    my favorite approach is mutation testing, I change the tested code and check whether test starts failing – gnat Sep 9 '17 at 18:46
  • 2
    @gnat - I believe there is some overlap but I do not feel it is the same question. Here I'm asking specifically about TDD and I believe that "If you are finding yourself in such situation then You are doing TDD wrong because ..." would be a valid answer. – AGrzes Sep 9 '17 at 18:59
  • 5
    it doesn't matter because you're not doing TDD already when you write test against already existing (and working correctly) implementation – gnat Sep 9 '17 at 19:10
  • 2
    "and what you do when you encounter that?" I run the new test (only this) with a coverage tool and check that the expected path is executed. Rolling back is only an option if you do the project for training. BTW: the Global Day of Coderetreat is a great opportunity to practice TDD... – Timothy Truckle Sep 9 '17 at 19:46
  • 2
    @Solomonoff'sSecret personally, I don't trust any test I haven't seen fail. – RubberDuck Sep 10 '17 at 22:52
13

Passing a test from the moment you wrote it could be a problem with your TDD process, but it doesn't mean it's wrong by itself.

Your test might pass by coincidence.

Let's say you have a method that returns the cost of withdrawing money using an ATM. You are now asked to add to it the new rule that if the card owner is over 60 years old, the cost is 0. So we test it, expecting it to fail:

assertTrue(ATM.withdrawalCost(clientOver60) == 0)

You might expect this to fail. But it passes, since client happens to be a VIP client, who have free withdrawals. Now you could go back to the withdrawalCost method and modify it to make it fail, but that's doesn't really make much sense. Write a new test to show your code is wrong:

assertTrue(ATM.withdrawalCost(nonVIPclientOver60) == 0)

Now it fails, you go and code until it passes, then repeat until done.

Should you erase the test then, since it makes no difference? No! It does describe the expected functionality of the ATM withdrawalCost method. If you erase it and someday the 0 cost withdrawal cost for VIP clients change, the first assert should still be true.

Having said this, for proper TDD you shouldn't go coding things ahead of your tests, and then testing the things you know you'll pass. I don't consider this the case you are asking about.

I believe the fail-code-pass cycle is meant to avoid the "I'll write this 3 tests that will fail and this 2 that will pass because I already coded it knowing what the test were going to be" driven development. You should know some people might feel otherwise. Hear their reasons, they might be valid too.

10

Tests passing right from the start will typically occur when one implemented something in a more general way than actually needed for the tests at hand. This is quite normal: unit tests can only provide a small, finite number of input values for a given function, but most functions are written for a huge range of possible input values. Often an implementation specifically designed for the current test cases would be more complicated than a more general solution. If that is the case, it would be cumbersome and error prone to artificially design the code in a way to work only for the test cases and fail for everything else.

For example, lets say you need a function to return the minimum of some values from a given array. You made an implementation, driven by a test with an array containing only one or two values. But instead of implementing this in a convoluted way by doing the comparisons on different elements (maybe only the first two elements), you call a minimum-of-array function from the standard library of your language ecosystem and so make the implementation a one liner. When you now decide to add a test with a five element array, the test will probably pass right from the start.

But how do you know then the test is not "green" because of a bug in the test itself? A simple and straightforward way to approach this is by making a temporary modification to the subject under test to make the test fail. For example, you could intentionally add a line if (array.size()==5) return 123 to your function. Now your five-element test will fail, so you know

  • the test is executed
  • the Assert call in the test is executed
  • the Assert call in the test validates the right thing

which should give you some confidence into the test. After you have seen the test fail, undo the modification, and the test should pass again.

Alternatively, you can modify the expected result of a test: lets say your passing test contains an assertion like

 int result = Subject.UnderTest(...);
 Assert.AreEqual(1,result);

then you can edit the test and replace the "1" by "2". When the test fails (as expected), you know it works as it should, and you can undo the replacement and see if the test now becomes green. The risk of introducing a bug into the test by such kind of replacement is very small, so this is probably acceptable for most real world cases.

A different, maybe debatable way, is to set a breakpoint into the test and use a debugger to step through it. That should also give you some confidence the test code is actually executed, and gives you the possibility to validate the path through the test by step-by-step inspection. However, one must be very careful not to overlook errors in a code path specificially for a failing test. For complex tests, you may consider to do both - making it artificially fail and use a debugger to inspect it.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.