Should the unit tests be independent of the implementations

Question

In the book- The Art of Unit Testing, I have read that the changes to the code base should not change the unit tests. But I have a hard time figuring out what should not be changed in the unit tests. Is it the behavior being tested or the code in the unit test? For me, changes to the test project code seems inevitable. As an example:

public interface IGreetingService
{
    void ShowGreeting();
}

public class ConsoleGreetingService : IGreetingService
{
    public void ShowGreeting()
    {
        Console.WriteLine("Hello!");
    }
}

public class MsgBoxGreetingService : IGreetingService
{
    public void ShowGreeting()
    {
        MessageBox.Show("Hello!");
    }
}

If we already have unit tests in place for the ConsoleGreetingService, we have to do some changes to the unit tests when we change our implementation of IGreetingservice to the MsgBoxGreetingService. It might be a simple change in the test code like changing the initialization of the ConsoleGreetingService object to MsgBoxGreetingService. But still, is that not considered a change in test or are changes like that considered "acceptable"? Then it seems like unit tests are not completely independent of the implementation code.

Answer 1

I have read that the changes to the code base should not change the unit tests.

Close, but no. Lets fix that:

Changes to the implementation choices, ones that change neither the behavior nor the API it's being tested through, should not change the unit tests.

There are many implementation choices that must be made in any codebase that can be changed without forcing changes to well designed unit tests. There's a whole book that talks about such choices. Refactoring by Martin Fowler.

A well designed unit test requires the code to exhibit a behavior and provide an API through which to elicit that behavior. Change either one and you break the unit test. But preserve that and the codebase is free to change as you like. Doing that is what Fowler called refactoring.

Your example code spectacularly fails to be unit testable code precisely because you provided no such API. This code is only testable with integration tests or obnoxious mocking.

A unit test should enforce expected behavior. The "behavior" code here is simply a hard coded string. One that is defined in multiple places. Sending that string to a console, a GUI or a print out is not a behavior. That's just a destination.

Writing unit testable code requires designing the code to be unit tested. A classic pattern for that is imperative shell, functional core. This pattern lets you push all the non-functional code away from your behavior code which will live in easily testable functional code that will provide that needed API.

This all boils down to writing code that does whatever logic to figure out what the greeting will be and unit testing that logic without knowing or caring where the greeting will go. Because that's IO. IO should not be part of the unit of a unit test. Push that out to the imperative shell. If you must test that (many don't) don't test it with a unit test.

Answer 2

I have read that the changes to the code base should not change the unit tests.

Let me say this straight: this makes no sense.

I don't have that book at hand, and you did not tell us which section lead you to this interpretation, still I am convinced you must have misread something. Unit tests (as well as any other kind of test) will not be required to change when an implementation of a certain (externally visible) behaviour changes, as long as the desired behaviour itself stays the same. But when the implementation changes up to the point where the desired behaviour changes, then the tests for this behaviour obviously have to change as well.

For example, different implementations of ConsoleGreetingService.ShowGreeting like

Console.WriteLine("Hello!");

or

const string greetings = "Hello!";
Console.WriteLine(greetings);

or

Console.WriteLine(GetHelloInCertainLanguage(new CultureInfo("en")) + "!");

can all be subject to the same, unchanged test which verifies the expected string is printed to the console.

Replacing a console output by some GUI output, however, is a change which obviously changes the externally visible behaviour, hence there is no reason to assume tests don't have to be changed.

_{Side note: how to implement a test which verifies that a certain message box with a certain text will be shown, and if it is justified call such a test a "unit test" is a separate topic which I don't discuss here.}

Answer 3

I have read that the changes to the code base should not change the unit tests.

The answer is that it depends.

If the change to the code base occurred early in the development process such that the code failed unit testing prior to the change, then no, you don't change your unit tests to make that failure a success. You change the code to make it pass the already written unit tests.

There are however occasions where changes to the code base do require changes to the unit tests. Unit tests are code, and like any other code, they can be buggy. If during the course of development, it is a unit test that turns out to be buggy, change that unit test. Do not reimplement the bug in the code base such that that the code base passes the buggy unit test.

Other occasions where a change to the unit tests is needed is when the code base change occurred during maintenance. Perhaps some external user reported what they think is a bug and the maintenance team concurs that it is a bug. Ideally, this bug should have been caught by unit testing prior to release. Changes to both the code base and to the unit tests are in order. Or perhaps external users requested a new feature, management concurred, and a prototype modification appears to be working. Once again, changes to both the code base and to the unit tests are in order.

Answer 4

I have read that the changes to the code base should not change the unit tests.

This 100000% makes sense and is critical to good unit testing.

The key thing, which I think you realise, is that the unit test should test the desired behaviour, which you don't expected to change regardless of implementation.

The goal of this rule is try and stop people writing unit tests which test the detail of implementations rather than the overall results of operations.

Your example is a good one in that both implementations hide what they do, one writes to console and one to messagebox, but they do it as a non-exposed, internal to the class thing. This type of thing would normally be considered part of the implementation, but in your case writing out the message is also the desired behaviour.

How do you follow the unit testing rule about not testing internal behaviour/implementation details whilst also testing the desired behaviour of writing messages? You have to change the class to make unit testing possible.

so for example you could inject the output writer

public interface IGreetingService
{
    void ShowGreeting();
}

public class GreetingService : IGreetingService
{
    IStreamWriter ouput
    public GreetingService(IStreamWriter ouput) { this.output = output }
    public void ShowGreeting()
    {
        ouput.WriteLine("Hello!");
    }
}

Now I can inject the message box or the console as required and the behaviour of the class is to write "Hello!" to the stream that is passed in. Which makes your unit test better.

TestGreeting()
{
    var testOutput = new Mock<IStreamWriter>();
    var target = new GreetingService(testOutput);
    target.ShowGreeting();
    var actual = testOutput.ReadLine();
    Assert(actual = "Hello!");
}

Now your unit test test the desired behaviour and won't change with the implementation. Because of the way you have defined "desired behaviour" and "implementation" for your class.

By following the rule you have been forced to change the design of your codebase. This is the intent of the rule, not to say that you will never have to rewrite your tests, but to try make you design your code in a such a way that your requirements are testable.

Also! there is an common anti pattern where people deliberately tests the internals of their classes and miss the real requirements. The rule tries to steer you away from doing this.

eg.

string Translate(string in)
{
   var out = "";
   var words = this.splitIntoWords(in);
   foreach(var w in words)
   {
      t = this.translate(w);
      out += w + " ";
   }
   return out
}

TranslateTest
{
   var t = new Translator();
   var out = t.Translate("hello world");
   Assert(t.splitIntoWords("hello world").WasCalled());
   Assert(t.translate("hello").WasCalled());
   Assert(t.translate("world").WasCalled());
}

Here our class is fully tested, however the actual requirements are not. It's not required that you split the string, or translate each word individually, that's just how you have chosen to implement the translation. The test will break if you change the implementation, even if the result is the same. Which we call "brittle", and in fact this test doesn't test the real requirement of was a correct translation returned.