11

Why is repeating yourself by writing tests so highly encouraged?

It seems that tests basically express the same thing as the code, and hence is a duplicate (in concept, not implementation) of the code. Wouldn't the ultimate target of DRY include elimination of all test code?

10 Answers 10

24

I believe this is a misconception any way I can think of.

The test-code that tests production-code is not at all similar. I'll demonstrate in python:

def multiply(a, b):
    """Multiply ``a`` by ``b``"""
    return a*b

Then a simple test would be:

def test_multiply():
    assert multiply(4, 5) == 20

Both functions have a similar definition but both do very different things. No duplicate code here. ;-)

It also occurs that people write duplicate tests essentially having one assertion per test function. This is madness and I have seen people doing this. This is bad practice.

def test_multiply_1_and_3():
    """Assert that a multiplication of 1 and 3 is 3."""
    assert multiply(1, 3) == 3

def test_multiply_1_and_7():
    """Assert that a multiplication of 1 and 7 is 7."""
    assert multiply(1, 7) == 7

def test_multiply_3_and_4():
    """Assert that a multiplication of 3 and 4 is 12."""
    assert multiply(3, 4) == 12

Imagine doing this for 1000+ effective lines of code. Instead you test on a per 'feature' basis:

def test_multiply_positive():
    """Assert that positive numbers can be multiplied."""
    assert multiply(1, 3) == 3
    assert multiply(1, 7) == 7
    assert multiply(3, 4) == 12

def test_multiply_negative():
    """Assert that negative numbers can be multiplied."""
    assert multiply(1, -3) == -3
    assert multiply(-1, -7) == 7
    assert multiply(-3, 4) == -12

Now when features are added/removed I only have to consider adding/removing one test function.

You may have noticed I have not applied for loops. This is because repeating some things is good. When I would have applied loops the code would be a lot shorter. But when an assertion fails it could obfuscate the output displaying an ambiguous message. If this occurs then your tests will be less useful and you will need a debugger to inspect where things go wrong.

  • 8
    One assertion per test is technically recommended because it means that multiple problems won't show as just a single failure. However, in practice, I think careful aggregation of assertions reduces the amount of repeated code and I almost never stick to one assert per test guideline. – Rob Church Jan 30 '14 at 10:44
  • @pink-diamond-square I see that NUnit does not stop testing after an assertion fails (which I think is weird). In that specific case it is indeed better to have one assertion per test. If a unit-testing framework does stop testing after a failed assertion multiple assertions are better. – siebz0r Jan 30 '14 at 12:12
  • 3
    NUnit doesn't stop the whole test suite, but that one test does stop unless you take steps to prevent it (you can catch the exception it throws, which is occasionally useful). The point I think they're making is that if you write tests that include more than one assert you won't get all the information that you need to correct the problem. To work through your example, imagine that this multiply function doesn't like the number 3. In this case, assert multiply(1,3) would fail but you wouldn't also get the failed test report about assert multiply(3,4). – Rob Church Jan 30 '14 at 12:50
  • I just thought I'd raise it because a single assert per test is, from what I've read in the .net world, the "good practice" and multiple asserts is "pragmatic usage". It looks a bit different in the Python documentation where the example def test_shuffle performs two asserts. – Rob Church Jan 30 '14 at 13:03
  • I agree and disagree :D There is clearly repetition here: assert multiply(*, *) == * so you could define an assert_multiply function. In the current scenario it does not matter by row count and readability, but by longer tests you can reuse complicated assertions, fixtures, fixture generating code, etc... I don't know whether this is a best practice, but I usually do this. – inf3rno Dec 3 '14 at 22:13
10

It seems that tests basically express the same thing as the code, and hence is a duplicate

No, this is not true.

Tests have a different purpose than your implementation:

  • Tests make sure that your implementation works.
  • They serve as a documentation: By looking at the tests, you see the contracts which your code must fulfil, i.e. which input returns what output, what are the special cases etc.
  • Also, your tests guarantee that as you add new features, your existing functionality does not break.
4

No. DRY is about writing code just once to do a particular task, test are validation that the task is being done correctly. It's somewhat akin to a voting algorithm, where obviously using the same code would be useless.

2

Wouldn't the ultimate target of DRY include elimination of all test code?

No, the ultimate target of DRY would actually mean elimination of all production code.

If our tests could be perfect specifications of what we want the system to do, we'd just have to generate the corresponding production code (or binaries) automatically, effectively removing the production code base per se.

This is actually what approaches like model-driven architecture claim to achieve - a single human-designed source of truth from which everything is derived by computation.

I don't think the reverse (getting rid of all tests) is desirable because :

  • You have to solve the impedance mismatch between implementation and specification. Production code can convey intent to a degree, but it will never be as easy to reason about as well-expressed tests. We human beings need that higher view of why we're building things. Even if you don't do tests because of DRY, specifications will probably have to be written down in documents anyway, which is a definitely more dangerous beast in terms of impedance mismatch and code desynchronization if you ask me.
  • While production code is arguably easily derivable from correct executable specifications (assuming enough time), a test suite is much harder to reconstitute from a program's final code. Specifications don't appear clearly just looking at the code, because interactions between code units at runtime are difficult to make out. This is why we have such a hard time dealing with testless legacy applications. In other words : if you want your application to survive for more than a few months, you'd probaly be better off losing the hard drive that hosts your production codebase than the one where your test suite is.
  • It's much easier to introduce a bug by accident in production code than in test code. And since production code is not self-verifying (though this can be approached with Design by Contract or richer type systems), we still need some external program to test it and warn us if a regression occurs.
1

Because sometimes repeating yourself is okay. None of these principles are meant to be taken in every circumstance without question or context. I have at times written tests against a naïve (and slow) version of an algorithm, which is a fairly clear-cut violation of DRY, but definitely beneficial.

1

Since unit-testing is about making unintentional changes harder, it can sometimes make intentional changes harder, too. This fact is indeed related to the DRY principle.

For example, if you have a function MyFunction which is called in production code in just one place, and you write 20 unit tests for it, you can easily end up having 21 places in your code where that function is called. Now, when you have to change the signature of MyFunction, or the semantics, or both (because some requirements change), you have 21 places to change instead of just one. And the reason is indeed a violation of the DRY principle: you repeated (at least) the same function call to MyFunction 21 times.

The correct approach for such a case is applying the DRY principle to your testing code as well: when writing 20 unit tests, encapsulate the calls to MyFunction in your unit tests in just a few helper functions (ideally just one), which are used by the 20 unit tests. Ideally, you end up with just two places in your code calling MyFunction: one from your production code, and one from you unit tests. So when you have to change the signature of MyFunction later, you will have only a few places to change in your tests.

"A few places" are still more than "one place" (what you get with no unit tests at all), but the advantages of having unit tests should heavily outweigh the advantage of having less code to change (otherwise you doing unit testing completly wrong).

0

One of the biggest challenges to building software is to capture requirements; that is to answer the question, "what should this software do?" Software needs exact requirements to accurately define what the system needs to do, but those who define the needs for software systems and projects often include people who do not have a software or formal (mathematics) background. The lack of rigor in requirements definition forced software development to find a way to validate software to requirements.

The development team found themselves translating the colloquial description for a project into more rigorous requirements. The testing discipline has coalesced as the checkpoint for software development, to bridge the gap between what a customer says they want, and what software understands they want. Both the software developers and the quality/testing team form understanding of the (informal) specification, and each (independently) write software or tests to ensure that their understanding conform. Adding another person to understand the (imprecise) requirements added questions and different perspective to further hone the precision of the requirements.

Since there has always been acceptance testing, it was natural to expand the testing role to write automated and unit tests. The problem was that meant hiring programmers to do testing, and thus you narrowed the perspective from quality assurance to programmers doing testing.

That all said, you are probably doing testing wrong if your tests differ little from the actual programs. Msdy suggestion would be to focus more on what in the tests, and less on how.

The irony is that rather than capture a formal specification of requirements from the colloquial description, industry has chosen to implement point tests as code to automate testing. Rather than produce formal requirements which software could be built to answer, the approach taken has been to test a few points, rather than approach building software using formal logic. This is a compromise, but has been fairly effective and relatively successful.

0

If you think your test code is too similar to your implementation code, this may be an indication that you are over-using a mocking framework. Mock-based testing at too low a level can end up with the test setup looking a lot like the method being tested. Try to write higher level tests that are less likely to break if you change your implementation (I know this can be hard, but if you can manage it you will have a more useful test suite as a result ).

0

Unit tests should not include a duplication of the code under test, as has been noted already.

I would add, though, that unit tests are typically not as DRY as "production" code, because setup tends to be similar (but not identical) across tests ... especially if you have a significant number of dependencies that you're mocking/faking.
It is of course possible to refactor this sort of thing into a common setup method (or set of setup methods) ... but I've found that those setup methods tend to have long parameter lists and be rather brittle.

So be pragmatic. If you can consolidate setup code without compromising maintainability, by all means do that. But if the alternative is a complex and brittle set of setup methods, a little bit of repetition in your test methods is OK.

A local TDD/BDD evangelist puts it this way:
"Your production code should be DRY. But it's OK for your tests to be 'moist'."

0

It seems that tests basically express the same thing as the code, and hence is a duplicate (in concept, not implementation) of the code.

This is not true, tests describe the use cases, while the code describes an algorithm which pass the use cases, so which is more general. By TDD you begin with writing use cases (probably based on the user story) and after that you implement the code necessary to pass these use cases. So you write a small test, a small chunk of code, and after that you refactor if necessary to get rid of the repetitions. That's how it works.

By tests there can be repetitions as well. For example you can reuse fixtures, fixture generating code, complicated assertions, etc... I usually do this, to prevent bugs in the tests, but I usually forget to test first whether a test really fails, and it can really ruin the day, when you are looking for the bug in the code for half an hour and the test is wrong... xD

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.