9

As I understand in theory under "unit" people mean method (in OOP). But in practice tests which verify some method in isolation are very fragile behaviour tests (verifying not the result but the fact that some dependency method was called). So I see a lot of people who by unit understand a small set of closely related classes. In this case only outer dependencies are mocked/stubbed and for dependencies which are inside unit real implementations are used. In this case there are more state, meaningful (according to specification) and not so fragile tests. So the question is how do you feel about these approaches and is it valid to call the second approach unit testing or may be it is some kind of low-level integration testing?

If you see some specific considerations about applying TDD by one of these ways of testing, I would be thankful for your thoughts.

4 Answers 4

11
+100

Originally TDD came from the agile movement, where testing was written up front as a way to ensure what you coded remained correct given the specification that was now well-defined in the test code. It also appeared as a very important aspect of refactoring, in that when you modified your code, you could rely on the tests to prove that you hadn't changed the code's behaviour.

Then the tools people came along and thought they knew information about your code and could then generate test stubs to assist you in writing your unit tests, and I think this is where it all went wrong.

The test stubs are generated by a computer that has no clue what it is you're doing, it just mindlessly produces a stub for every method because that's what it's told to do. This means you have a test case for each method regardless of the complexity of that method or whether its suitable for testing in isolation.

This is coming at testing from the wrong end of the TDD methodology. In TDD you're supposed to figure out what the code is to do, and then produce code that achieves this. This is self-fulfilling in that you end up writing tests that prove the code does what the code does, not what its supposed to do. Combined with the auto-generation of method-based test stubs, you pretty much waste your time proving each tiny aspect of your code that can so easily prove to be wrong when all the little pieces are put together.

When Fowler described testing in his book, he referred to testing each class with its own main method. He improved this, but the concept is still the same - you test the entire class so it works as a whole, all your tests are bundled together to prove the interaction of all those methods so the class can be reused with defined expectations.

I think the test toolkits have done us a disservice, led us down the path of thinking that the toolkit is the only way to do things when really, you need to think more for yourself to get the best result from your code. Blindly putting test code in test stubs for tiny pieces just means you have to repeat your work in an integration test anyway (and if you're going to do that, why not skip the now-redundant unit test stage completely). It also means people waste a lot of time trying to get 100% test coverage, and a lot of time creating large amounts of mocking code and data that would have been better spent making the code easier to integration test (ie if you have that much data dependencies, unit test might not be the best option)

Lastly, the fragility of method-based unit tests just shows up the problem. Refactoring is designed to be used with unit tests, if your tests break all the time because you're refactoring then something has gone seriously wrong with the whole approach. Refactoring likes to create and delete methods, so obviously the blind per-method based test approach is not what was originally intended.

I have no doubts that many methods will get tests written for them, all of the public methods of a class should be tested, but you cannot get away from the concept of testing them together as part of a single test case. For example, if I have a set and a get method, I can write tests that put the data in and check the internal members are set ok, or I can use each to put some data and then get it out again to see if it's still the same and not garbled. This is testing the class, not each method in isolation. If the setter relies on a helper private method, then that's fine - you don't need to mock the private method to ensure the setter is working, not if you test the entire class.

I think the religion is getting into this topic, hence you see the schism into what's now known as 'behaviour-driven' and 'test-driven' development - the original concept of unit testing was for behaviour driven development.

10

A unit is most commonly defined as "the smallest testable part of an application". More often than not, yes, this means a method. And yes, this means that you shouldn't test the result of a dependant method, but just that the method is called (and then only once, if possible, not in every test for that method).

You call this fragile. I think that is incorrect. Fragile tests are those that break under the slightest change to unrelated code. That is, those which rely on code that is irrelevant to the functionality being tested.

However, what I think you really mean to say is that testing a method with none of its dependencies is not thorough. On that point I would agree. You also need integration tests to make sure that the units of code are correctly hooked together to make an application.

This is exactly the problem that behaviour-driven development, and specifically acceptance-test-driven development, sets out to solve. But that doesn't remove the need for unit tests / test-driven development; it merely complements it.

5
  • I really wanted to say behaviour tests are fragile. They often become false negative during codebase change. It happens less with state tests (but state tests are very seldom for unit testing) Commented Aug 14, 2011 at 22:15
  • @Idsa: I'm a little lost by your definitions. Behaviour tests are integration tests, testing a piece of behaviour as specified. Reading your original question, it seems that when you say state tests, you mean the same thing.
    – pdr
    Commented Aug 14, 2011 at 22:26
  • by state I mean test which verifies the state, result of some function; by behavour I mean test which verifies not the result, but the fact that some function was called Commented Aug 15, 2011 at 7:26
  • @Idsa: In that case, I completely disagree. What you are calling state test, I call integration. What you are calling behaviour, I call unit. Integration tests by their very definition are more fragile. Google "integration test unit fragile" and you'll see that I'm not alone.
    – pdr
    Commented Aug 15, 2011 at 10:09
  • there are a log of articles about testing but which of them share your opinion? Commented Aug 17, 2011 at 19:18
2

As the name suggests you are testing an atomic subject in each test. Such a subject is usually a single method. Multiple tests can test the same method, in order to cover the happy path, possible errors, etc. You are testing behaviour, not internal mechanics. So unit testing is really about testing a class' public interface, i.e. a specific method.

In unit testing, a method needs to be tested in isolation, that is, by stubbing/mocking/faking any dependencies. Otherwise, testing a unit with 'real' dependencies makes it an integration test. There is a time and place for both types of tests. Unit tests ensure a single subject works as expected, standalone. Integration tests ensure that 'real' subjects work together correctly.

1
  • 1
    not quite, a unit is any isolated subject, just because the automated tooling prefers to treat a method as this doesn't make it so, nor make it the best. 'Isolated' is the key here. Even if you test methods, you should also test the private ones too.
    – gbjbaanb
    Commented Mar 7, 2013 at 14:56
1

My rule of thumb: The smallest unit of code which is still complex enough to contain bugs.

Whether this is a method or class or subsystem depends on the particular code, no general rule can be given.

For example, it does not provide any value to test simple getter/setter methods in isolation, or wrapper methods which only call another method. Even a whole class might be too simple to test, if the class is only a thin wrapper or adapter. If the only thing to test is if a method on a mock is called, then the code under test is to thin.

In other cases a single method might perform some complex calculation which is valuable to test in isolation.

In many cases the complex parts are not individual classes but rather the integration between classes. So you test two or more classes at once. Some will say that this is not unit tests but integration tests, but never mind the terminology: You should test where the complexity is, and these tests should be part of the test suite.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.