5

I have two test cases to test a fix to code that is using the wrong criteria for selecting objects from a collection:

  1. Given one object in the collection matching the bad criteria, ensure that no object is returned.
  2. Given two objects in the collection, with the later one matching the bad criteria, ensure that the earlier object is retrned.

Right now, both of these tests will fail.

The problem is, as soon as I make the fix, both tests will pass.

Is this ok? I thought we needed a single failing unit test before writing the code. I haven't heard it's ok to have multiple failing unit tests before writing code.

Improve this question
5
  • 6
    "I thought we needed a single failing unit test before writing the code."? Why? Where did you see this? Can you provide a link, quote or reference? You need at least one. Why do you want it to be only one?
    – S.Lott
    Dec 28, 2011 at 19:44
  • @S.Lott: no, not really. That's what I took from reading the "classic" TDD book some ages ago (2004).
    – John Saunders
    Dec 28, 2011 at 19:45
  • 1
    "the "classic" TDD book"? What book?
    – S.Lott
    Dec 28, 2011 at 19:47
  • my.safaribooksonline.com/book/…
    – John Saunders
    Dec 28, 2011 at 22:47
  • This? "The TDD cycle is as follows. 1. Add a little test. 2. Run all tests and fail. 3. Make a change. 4. Run the tests and succeed. 5. Refactor to remove duplication." Is that where the "Single Test" idea came from? The word single only occurs in a few places in the book. None of which seem to include "single test".
    – S.Lott
    Dec 29, 2011 at 2:05

4 Answers 4

Reset to default
11

I thought we needed a single failing unit test before writing the code. I haven't heard it's ok to have multiple failing unit tests before writing code.

That's for writing new features, not for fixing bugs, and only if you want to practice ultra-orthodox TDD.

Secondly, an ideal test suite would be one where any bug causes exactly one test to fail, because that makes it easiest to pinpoint where the bug occurs. But this is an ideal that one can aspire to but hardly ever reach, and that's OK. Try to make your unit tests orthogonal if possible, but don't worry if you can't do it all the time.

Improve this answer
1
  • 2
    Actually, I find orthogonality very hard to achieve. Most of my unit tests (at least for non-trivial functions) end up testing first the most rudimentary constraints (e.g., must throw if passed NULL), and then more complex situations; quite regularly, the complex ones depend on preconditions checked by the simpler ones, so if the simple one fails, so does the complex one (but not vv.)
    – tdammers
    Dec 28, 2011 at 21:32
18

No, the point of the test suite is that (as far as you are able) it covers every conceivable thing that code changes might break. It isn't that important whether one or two tests fail - the point is, the change broke something, and it can't be allowed in before it passes all tests. There is absolutely no point going to contortions and trying to make the correction in two separate steps, just to fix exactly one test per commit - it's just extra effort, and I dare anyone to explain to me what it buys you.

Improve this answer
6

There is no rule that says you should make only a single test pass at a time.

Usually you don't write too many tests before any code, but you can.

Think of them as your specification.

Here are really great video tutorials on TDD:

Funq: screencast series on how to build a DI container using TDD

Improve this answer
1

In our test cases we try to write them in 3 sections (where appropriate)

  1. Setup
  2. Test Execution
  3. Assertion

Sometimes section might be missing but in overall they cover most of the test cases.

In your case it's perfectly OK for a few test to fail if feature doesn't work - the main thing will be to understand the difference and similarity (what different is tested and why tests are failing together). As a solution we've chosen a proper test naming convention that is:

MethodBeingTested_ParametersOrTestCriteria_ExpectedResult()

Then your two tests would be named like

Method1_1BadObject_ReturnsNoObjects()
Method1_1BadObjectAnd1GoodObject_ReturnsGoodObject()

I know that using underscores is a tough decision but it works just perfect for us. You can just glance at the failing test and identify what was expecting and what is failing.

If they both fail it is clear that Method1 is the reason.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.