4

Most (all?) TDD resources show you how the cycle goes:

  • Write Test
  • Check Test
  • Write Production Code
  • Check Test
  • Clean up Production Code
  • Check test

They also - to me - seem to imply that all code is covered by tests.

However, reading posts like Writing Great Unit Tests: Best and Worst Practices (for example, linked to here) it's said that

TDD is a robust way of designing software components (“units”) interactively so that their behaviour is specified through unit tests

To fill in some more context:

At the other end of the scale, integration tests contain no knowledge about how your codebase is broken down into units, but instead make statements about how the whole system behaves towards an external user.

This makes perfect sense to me. However, it implies that large parts of the applications code are not covered by tests. Why? Because if you have units (and you need a lot of units to get your Unit Tests right) you need code that wires the units together. This code, IMHO, will get complicated enough that it deserves to be tested on a more granular level that integration tests while it probably falls into "Dirty Hybris":

Anywhere in between, it’s unclear what assumptions you’re making and what you’re trying to prove. Refactoring might break these tests, or it might not, regardless of whether the end-user experience still works

So to sum up:

  • I easily see the value of Unit Tests with TDD
  • Some code is needed to wire the units together
  • This code will be complex and integration testing will/may not be enough.
  • TDD resources on the net seem to imply 100% code/test coverage, yet
  • looking at TDD with True Unit Tests will leave some code untested

Edit: wikipedia says:

Integration testing takes as its input modules that have been unit tested

but somehow I feel there's still quite a bit of wiring code missing from the picture. (except if "module" means "class or function" because that is what's unit tested in isolation)


Insights?

  • 3
    Perhaps the wikipedia quote would be clearer if it was formulated like this: Integration testing takes as its input units that have been unit tested. Yes, module can be interpreted as meaning class or function in this case. – Treb Oct 4 '11 at 15:06
  • You may have forgotten that you can also unit-test the glue that wires units together - a unit doesn't have to live in a vacuum to be tested; wiring it up against mock classes or simulating whatever ecosystem they need is perfectly fine, as long as the tests test what they should test. – tdammers Oct 4 '11 at 15:31
8

However, it implies that large parts of the applications code are not covered by tests. Why? Because if you have units (and you need a lot of units to get your Unit Tests right) you need code that wires the units together. This code, IMHO, will get complicated enough that it deserves to be tested on a more granular level that integration tests while it probably falls into "Dirty Hybris":

Your assumption is faulty because you are neglecting a layer of testing - acceptance testing.

Your unit tests cover individual units - the classes and methods that compose them. This enables you to test methods and classes in isolation to ensure that they are behaving as expected. Above this lies your integration tests, which tests the collaboration between classes and ensures that larger modules (packages and even inter-package collaboration) work as expected. Finally, your acceptance tests are used to verify and validate your entire system, as assembled, against the user requirements.

Assuming that you have the appropriate unit and integration tests that correspond to requirements and well-defined acceptance criteria and acceptance test plans, then everything in your system is tested. Other aspects of testing - smoke tests, regression tests, and so forth, are simply an appropriate subsampling of the unit, integration, and acceptance tests.

TDD is a robust way of designing software components (“units”) interactively so that their behaviour is specified through unit tests

That particular quote is also missing something. As I was taught, TDD isn't just about unit tests, but developing all tests first. That includes not only unit tests, but the necessary acceptance and integration tests as well.

  • You say: "unit tests ... above this lies your integration tests" -- however, the post I link to says "Integration Tests: Automate the entire system" ... this doesn't sound as if integration tests lie above unit tests. – Martin Ba Oct 4 '11 at 13:35
  • 3
    @Martin: From the linked post: If you automate this sort of testing in order to detect breakages when they happen in the future, it’s called integration testing. AFAIK, this statement is wrong, this is the definition of regression testing. Integration testing is exactly as Thomas says, one level above unit testing (you test how well your units integrate into a system). – Treb Oct 4 '11 at 13:57
  • @Martin That post gets a few definitions and concepts, including TDD and integration testing, wrong, at least based on the standard definitions that I've learned. – Thomas Owens Oct 4 '11 at 14:08
  • 1
    +1 thomas - this is an extremely common misconception; TDD tests scale, unit through acceptance, because they test features, not modules. Should never have called them 'unit' tests, too confusing! – Steven A. Lowe Oct 5 '11 at 3:00
0

There are two different kinds of test coverage:

  • Code coverage, and
  • Case coverage

Code coverage is a measure of what percent of the code is covered by an automated test, whereas case coverage is a measure of what percent of user requirements are covered by tests.

Applications covered under TDD using unit tests as a foundation also tend to include functional, integration, and UI/e2e tests which cover user requirement cases. This gives the application multiple layers of testing.

You make an erroneous assumption in the question: "This code will be complex and integration testing will/may not be enough."

That could be the case if there are no clear boundaries between pure code units and units with side-effects. If you isolate side-effects from program logic, and break units down into independent, composable pieces (atomic units of composition), those units can then be composed declaratively with very little connective logic.

e.g., You can use functions like this to compose the units together (JavaScript example):

const compose = (...fns) => x => fns.reduceRight((y, f) => f(y), x);

You use that function like this:

const hashObject = compose(
  serialize,
  stringToBuffer,
  bufferToHash,
  hashToString
);

You can compose asynchronous functions which have I/O dependencies similarly as well, with a nearly identical API.

As you can see, no complex, imperative logic to test. If you know these compositional tools have their own unit tests and you trust them, all you'll need to write is an integration test.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.