Writing the minimum code to pass a unit test - without cheating!

Question

When doing TDD and writing a unit test, how does one resist the urge to "cheat" when writing the first iteration of "implementation" code that you're testing?

For example:
Let's I need to calculate the Factorial of a number. I start with a unit test (using MSTest) something like:

[TestClass]
public class CalculateFactorialTests
{
    [TestMethod]
    public void CalculateFactorial_5_input_returns_120()
    {
        // Arrange
        var myMath = new MyMath();
        // Act
        long output = myMath.CalculateFactorial(5);
        // Assert
        Assert.AreEqual(120, output);
    }
}

I run this code, and it fails since the CalculateFactorial method doesn't even exist. So, I now write the first iteration of the code to implement the method under test, writing the minimum code required to pass the test.

The thing is, I'm continually tempted to write the following:

public class MyMath
{
    public long CalculateFactorial(long input)
    {
        return 120;
    }
}

This is, technically, correct in that it really is the minimum code required to make that specific test pass (go green), although it's clearly a "cheat" since it really doesn't even attempt to perform the function of calculating a factorial. Of course, now the refactoring part becomes an exercise in "writing the correct functionality" rather than a true refactoring of the implementation. Obviously, adding additional tests with different parameters will fail and force a refactoring, but you have to start with that one test.

So, my question is, how do you get that balance between "writing the minimum code to pass the test" whilst still keeping it functional and in the spirit of what you're actually trying to achieve?

Answer 1

It's perfectly legit. Red, Green, Refactor.

The first test passes.

Add the second test, with a new input.

Now quickly get to green, you could add an if-else, which works fine. It passes, but you are not done yet.

The third part of Red, Green, Refactor is the most important. Refactor to remove duplication. You WILL have duplication in your code now. Two statements returning integers. And the only way to remove that duplication is to code the function correctly.

I'm not saying don't write it correctly the first time. I'm just saying it's not cheating if you don't.

Answer 2

Clearly an understanding of the ultimate goal, and the achievement of an algorithm that meets that goal, is required.

TDD is not a magic bullet for design; you still have to know how to solve problems using code, and you still have to know how to do that at a level higher than a few lines of code to make a test pass.

I like the idea of TDD because it encourages good design; it makes you think about how you can write your code so that it is testable, and generally that philosophy will push the code towards a better design overall. But you still have to know how to architect a solution.

I do not favor reductionist TDD philosophies that claim you can grow an application by simply writing the smallest amount of code to pass a test. Without thinking about architecture, this won't work, and your example proves that.

Uncle Bob Martin says this:

If you're not doing Test Driven Development, it's very difficult to call yourself a professional. Jim Coplin called me on the carpet for this one. He didn't like that I said that. In fact, his position right now is that Test Driven Development is destroying architectures because people are writing tests to the abandon of any other kind of thought and tearing their architectures apart in the mad rush to get tests to pass and he's got an interesting point, that's an interesting way to abuse the ritual and lose the intent behind the discipline.

if you're not thinking through the architecture, if what you're doing instead is ignoring architecture and throwing tests together and getting them to pass, you're destroying the thing that will allow the building to stay up because it's the concentration on the structure of the system and solid design decisions that help the system maintain its structural integrity.

You cannot simply just throw a whole bunch of tests together and make them pass for decade after decade after decade and assume that you're system is going to survive. We don't want to evolve ourselves into hell. So a good test driven developer is always conscious of making architectural decisions, always thinking of the big picture.

Answer 3

A very good question...and I have to disagree with almost everyone except @Robert.

Writing

return 120;

for a factorial function to make one test pass is a waste of time. It's not "cheating", nor is it following red-green-refactor literally. It is wrong.

Here's why:

• Calculate Factorial is the feature, not "return a constant". "return 120" is not a calculation.

• the 'refactor' arguments are misguided; if you have two test cases for 5 and 6, this code is still wrong, because you are not calculating a factorial at all:

  if (input == 5) { return 120; } //input=5 case
  else { return 720; }   //input=6 case
  

• if we follow the 'refactor' argument literally, then when we have 5 test cases we would invoke YAGNI and implement the function using a lookup table:

  if (factorialDictionary.Contains(input)) {
      return factorialDictionary[input]; 
  }
  throw new Exception("Input failure");
  

None of these are actually calculating anything, you are. And that's not the task!

Answer 4

When you've written only one unit test, the one-line implementation (return 120;) is legitimate. Writing a loop calculating the value of 120 - that would be cheating!

Such simple initial tests are a good way to catch edge cases and prevent one-off errors. Five actually isn't the input value I'd start with.

A rule of thumb that could be useful here is: zero, one, many, lots. Zero and one are important edge cases for the factorial. They can be implemented with one-liners. The "many" test case (e.g. 5!) would then force you to write a loop. The "lots" (1000!?) test case could force you to implement an alternative algorithm to handle very large numbers.

Answer 5

As long as you only have a single test, then the minimal code needed to pass the test is truly return 120;, and you can easily keep it for that as long as you don't have any more tests.

This allows you to postpone further design until you actually write the tests that exercise OTHER return values of this method.

Please remember that the test is the runnable version of your specification, and if all that specification says is that f(6)=120 then that fits the bill perfectly.

Answer 6

If you are able to "cheat" in such a way, it suggests that your unit tests are flawed.

Rather than testing the factorial method with a single value, test it was a range of values. Data-driven testing can help here.

View your unit tests as a manifestation of the requirements - they must collectively define the behaviour of the method which they test. (This is known as behaviour driven development - its the future ;-))

So ask yourself - if someone were to change the implementation to something incorrect, would your tests still pass or would they say "hang on a minute!"?

Bearing that in mind, if your only test was the one in your question, then technically, the corresponding implementation is correct. The problem is then viewed as poorly-defined requirements.

Answer 7

Just write more tests. Eventually, it would be shorter to write

public long CalculateFactorial(long input)
{
    return input <= 1 ? 1 : CalculateFactorial(input-1)*input;
}

than

public long CalculateFactorial(long input)
{
    switch (input) {
       case 0: return 1;
       case 1: return 1;
       case 2: return 2;
       case 3: return 6;
       case 4: return 24;
       case 5: return 120;
    }
}

:-)

Answer 8

Writing "cheat" tests is OK, for sufficiently small values of "OK". But recall - unit testing is only complete when all tests pass and no new tests can be written that will fail. If you really want to have a CalculateFactorial method that contains a bunch of if statements (or even better, a big switch/case statement :-) you can do that, and since you're dealing with a fixed-precision number the code required to implement this is finite (although probably rather large and ugly, and perhaps limited by compiler or system limitations on the maximum size of a procedure's code). At this point if you really insist that all development has to be driven by a unit test you can write a test that requires the code to compute the result in an amount of time shorter than that which can be accomplished by following all the branches of the if statement.

Basically, TDD can help you write code which implements requirements correctly, but it can't force you to write good code. That's up to you.

Share and enjoy.

Answer 9

I do 100% agree with Robert Harveys suggestion here, it's not just about making tests pass, you need to keep the overall goal in mind too.

As a solution to your painpoint of "it's only verified to work with a given set of inputs" I'd propose using data driven tests, such as xunit theory. The power behind this concept is that it allows you to easily create Specifications of inputs to outputs.

For Factorials, a test would look like this:

    [Theory]
    [InlineData(0, 1)]
    [InlineData( 1, 1 )]
    [InlineData( 2, 2 )]
    [InlineData( 3, 6 )]
    [InlineData( 4, 24 )]
    public void Test_Factorial(int input, int expected)
    {
        int result = Factorial( input );
        Assert.Equal( result, expected);
    }

You could even implement a test-data provide (that returns IEnumerable<Tuple<xxx>>) and encode a mathematic invariant, such as repeatedly dividing by n will yield n-1).

I find this tp be a very powerful way of testing.

Answer 10

If you still able to cheat then the tests are not enough. Write more tests! For your example, I will try to add tests with input 1, -1, -1000, 0, 10, 200.

Nevertheless, if you're really commit to cheat you can write an endless if-then. In this case, nothing could help except code review. You would be soon caught on acceptance test (written by other person!)

The problem with unit tests is sometimes programmers look at them as unnecessary work. The correct way to see them is as tool for you to make the result of your work correct. So if you create an if-then, you know unconsciously that there are other cases to consider. This means you have to write another tests. And so on and so on until you realize the cheating is not working and it is better to just code the correct way. If you still feel that you are not finish, you are not finish.

Answer 11

I would suggest that your choice of test is not the best test.

I would start off with:

factorial(1) as the first test,

factorial(0) as the second

factorial(-ve) as the third

and then continue on with non-trivial cases

and finish up with an overflow case.

Answer 12

The fallacy here in this example is that the library method under test will contain something simple and well-known. That the amount of work needed to write the correct production-ready method is about the same as the amount of work needed to just make the test pass. For any non-trivial piece of code that is simply not true.

This is actually not true even for factorials because it takes longer to calculate them than to look them up in an array for example, but what if you don't have a pre-calculated value in your array? Can you work with built-in native datatypes or do you need greater precision which requires using a custom data type? Should it be able to handle fractional values when you need the gamma function? What should happen if an invalid number is passed?

All these things would require you to come back later, and fix the initial implementation so what you consider the correct solution to implement at first glance, may not actually be the one required! And it may take longer to implement than you expect, especially when you consider corner cases and production quality error handling. And this is just for something as simple and well-defined as a factorial.

By hardcoding just the values needed by your current test suite, you can postpone all this work for later when you know the needs better, and work on actually getting the rest of tests for what you work on to run.

Answer 13

There is a saying: We should keep an open mind, but not so open that our brain falls out.

Not writing a fully working factorial function is in the “your brain falls out” category. The argument “but but we need to check that wrong return values produce test failures” is nonsense - I do this by a two character change returning n! + 1. That has the advantage that I can test the tests in a meaningful way, in case we have a test “5! must be between 100 and 150”. And if you have created an implementation where returning n! + 1 is difficult then figure out why it is difficult and fix it.

Where it makes sense: You write something a bit more difficult. Like translating a string to title case. That’s where you can reasonably start with code that works with a single word of ASCII letters, then multiple or zero words, then things like “john’s” -> “John’s” and not “John’S” and so on. And in the last case for example, I’d expect you to not look for an apostrophe only in your code, but all kinds of characters that cause this behaviour.