25

I read Stephen Walther's article about Test Driven Development (TDD) and unit testing.

I think it was an excellent article.

The author makes a distinction between what he calls "TDD Tests" and unit testing. They appear to be different tests to him.

Previous to reading this article I thought unit tests were a by-product of TDD. I didn't realise you might also create "TDD tests".

The author seems to imply that creating unit tests is not enough for TDD as the granularity of a unit test is too small for what we are trying to achieve with TDD. So his TDD tests might test a few classes at once.

At the end of the article there is some discussion from the author with some other people about whether there really is a distinction between "TDD Tests" and unit testing. Seems to be some contention around this idea.

The example "TDD tests" the author showed at the end of the article just looked like normal MVC unit tests to me - perhaps "TDD tests" vs unit tests is just a matter of semantics?

I would like to hear some more opinions on this, and whether there is / isn't a distinction between the two tests.

6
  • 1
    "just a matter of semantics"? Are you saying that "semantics" == "irrelevant" or "meaningless" or "too subtle" or something like that. It's a bad phrase to use, since your question is entirely about the semantics of the words and nothing more.
    – S.Lott
    Commented Jun 24, 2011 at 2:39
  • 1
    @S.Lott Yeah, bad phrase. I just mean to say that it's not clear in the author's "TDD Test" example how that differs from how one would normally unit test the same MVC controller class.
    – asgeo1
    Commented Jun 24, 2011 at 3:49
  • 1
    Interesting question, I must say I finished reading that article feeling his argument was pretty flaky/not well structured. Apart from saying that they should be named differently based on whether test-first was used or not, there was no compelling distinction between the two. I'd say just concentrate on writing the tests, whatever you want to call them.
    – mjhilton
    Commented Jun 24, 2011 at 6:55
  • 1
    (the amazing banner/logo didn't do much for my perception of his credibility, mind you)
    – mjhilton
    Commented Jun 24, 2011 at 6:58
  • 1
    @mjhilton: yeah, not clear what's going on there. Is he being attacked by a hairy cephalophagic Pac-Man? Commented Jun 24, 2011 at 13:07

8 Answers 8

15

I like TDD video by Roy Osherove especially the part about 'What does TDD mean?'.

It all depends on what you 'drive' by TDD (design or development, or both or ...) and what the 'unit', that is under test, really is.

The term Unit Testing has become overloaded. Programmer tests might be more precise and might help clarify better the role this kind of testing plays in TDD practice. I think he is really comparing programmers tests and the tests that cover more than one class/method/module.

I don't think that TDD practice exclude any type of testing to be the 'T'. It is a practice and it is not really a type of test (as someone pointed out in the comments).

However, TDD has the most obvious effect at the lowest level (at method/function level) hence programmers tests are most often used and indirectly assumed when talking about TDD. But TDD as a practice is still quite possible when you move up (pre-integration or integration tests) as well. It just becomes a bit more difficult to quickly link the failing tests with the line(s) of code that needs to be added/modified.

5
  • 3
    The term Unit Testing is not overloaded, it's just abused by people who don't know what they're talking about. Commented Feb 5, 2013 at 16:47
  • 1
    @SnOrfus People having a definition different from yours can say the same thing. Commented Feb 5, 2013 at 17:42
  • 2
    @Simon: That doesn't make me wrong, or them correct. If I claim tomorrow that "orange" is word describing the colour of mcintosh apples; it doesn't make me a person with a different definition, it makes me ill-informed or confused but ultimately just incorrect. Commented Feb 5, 2013 at 18:45
  • 1
    @SnOrfus I agree with you. But I think that the unit test concept is newer than the one of the Orange color, and that sometimes the meaning of a given word escapes from its original definition because people start to use it differently. At this point, it is ok to say that the term is overloaded (even for bad reasons). In this case, being new to tdd / unit tests, I find it hard to define what should be considered as a unit of code. So your comment made me smile & react :-). Of course that does not mean that you are wrong about your definition. Commented Feb 5, 2013 at 18:57
  • @simon - "People having a definition different from yours can say the same thing". I'll probably go with the IEEE definition thanks. 1008-1987 - IEEE Standard for Software Unit Testing
    – Deleted
    Commented Aug 8, 2014 at 15:31
7

TDD tests Features, not "units"

sometimes features correspond to units, sometimes they don't. Usually they don't.

TDD tests scale - up or down - as required to test a feature. If a feature is 'must transmit daily updates in less than 500msec' then the TDD test will look like a unit test. If a feature is 'must construct correct widget prototype bindings with six optional branch structures' then the TDD test will look like an acceptance test. [features paraphrased, obviously]

Caveat: didn't read article; read Beck

3
  • "TDD tests Features, not units" - Incorrect. If one follows the TDD mantra, then TDD drives the code and any code you write has an equivalent test. Therefore one net outcome of TDD is theoretical 100% code coverage (though in the real world I would challenge that). TDD tests "units" and assuming you follow the religion correctly will have tested your entire app
    – Deleted
    Commented Aug 8, 2014 at 15:38
  • @MickyDuncan: a common misconception. code coverage is independent of TDD. See for example martinfowler.com/bliki/TestCoverage.html Commented Aug 8, 2014 at 18:33
  • Incorrect. Mr Fowler is talking about the difference between coverage and quality. Anyway, if you want both you should take a leaf from the electronics field and have your QA team use Model-based testing. We should not be relying on developer-focused testing to test coverage let alone quality. DEV-level testing serves a purpose but it is not objective. TDD is dead. Long live testing.
    – Deleted
    Commented Aug 8, 2014 at 23:49
4

Even though I have no experience with TDD, I will add my own opinion on this ;-)

The article says that the approach of TDD leads to tests that may not adhere to the principles of a Unit Test, specifically testing in isolation. You could say that unit tests are more bottom-up, while TDD-Tests, coming from the user stories, have a top-down approach.

I think that

  1. He is right about that
  2. He makes a poor job of explaining it
  3. If you see it happen, you should refactor your tests. Don't give them a different name (e.g. TDD-Tests) and feel fancy because of it, but turn them into proper Unit Tests.
0
3

FWIW, I use the term "TDD test" to refer to the tests that come out of the TDD process, and "unit test" to refer to those tests that would verify that a "unit" meets its specification. The two aren't exactly the same - a TDD test (the way I use the term) always precedes an addition of code to the unit under test, but such tests are not necessarily adequate to the verification of a module. Consider a trivial function like isBetween - given a range and a number, verify that the number falls within the range. Implementing with TDD, we might come up with:

# first pass
def test_LowerMeansNotInBetween
    # zero is not between 2 and 4
    assert_false(isBetween(2, 4, 0))
end
def isBetween(low, high, testVal)
    false
end
# second pass
def test_BetweenMeansInBetween
    assert(isBetween(2, 4, 3))
end
def isBetween(low, high, testVal)
    low <= testVal
end
# third pass
def test_GreaterMeansNotInBetween
    assert_false(isBetween(2, 4, 5))
end
def isBetween(low, high, testVal)
    (low <= testVal) && (testVal <= high)
end

At this point, isBetween should be functionally correct (but I haven't actually tried running it, I could be wrong -- this is just for sake of illustration), but the tests do not fully exercise the specified behavior. For that, we might need at least the following tests:

def test_AtLowerBoundMeansBetween
    assert(isBetween(2, 4, 2))
end
def test_AtUpperBoundMeansBetween
    assert(isBetween(2, 4, 4))
end

which didn't cause us to update any code. The way I see it, all five tests are "unit tests", but only the first three are "TDD tests".

4
  • I read an article from Uncle Bob, in which he suggests that their might be a right order in which we should "transform" our code and tests step by step. Maybe the answer to your issue lays in "the choice of the next test to write". And there might be a strategy to find it and avoid the issue you are describing. See this article : cleancoder.posterous.com/the-transformation-priority-premise
    – David
    Commented Jun 24, 2011 at 11:41
  • @David: I read that article, but I don't think it addresses the concern. Basically, I see "unit tests" as providing an operational specification of the unit's expected behavior, while "TDD tests" are tests written following the TDD process. I've noticed a few times that, after following the TDD process for a while, the resulting code is complete, but the tests don't fully describe the intended behavior - there's room for bugs in the code. At this point, we can write unit tests to cover those remaining cases, and we don't expect to have to change the code under test unless bugs are discovered. Commented Jun 24, 2011 at 14:38
  • maybe you're right, or maybe we're missing the "path" or "strategy" that will allow us to have complete tests. At some point in your example, the (production) code is doing more than what is tested for. The code seems to have taken a step too far. Maybe there was an other test that should have been written first. The question is "is it possible, using a deterministic method, to find that test ?". This article doesn't give the answer, but it clearly tells us that there are strategies to find.
    – David
    Commented Jun 25, 2011 at 14:13
  • Good answer. I'm actually a little flabbergasted, because I thought it was really explicit that TDD means "write the tests before you write code". But I see people use TDD synonymously with unit testing. Which misses the whole point of the methodology. But there's a lot of confusion about unit testing in general. For example, I've worked with lots of folks who don't see (and don't care about) the distinction between unit & integration tests. Hm. And reading the comments to the linked article, there is tons of confusion about what "unit" means. Arg.
    – sea-rob
    Commented Mar 13, 2014 at 5:26
2

as @Steven said, TDD tests should test features, or more generally they test use cases. Unit tests have a different, smaller scope: test the class implementation from its interface as a single unit of delivery; it constitutes an acceptance test for a single class

think of TDD as producing a set of acceptance tests that are required for a certain sprint on a whole project, so the purpose of the sprint is making the acceptance tests pass for all the project running as a single unit.

the set of all use case tests, or acceptance tests is a guarantee that the software does what the stakeholder has asked the team to implement

1

YES

Unit test alone is basically a method for testing the smallest logical part of a program.

TDD is a software development process. It basically tell you how to use unit test methods in a more agile way.

In other words, unit test answer the questions "what to do?" and TDD answers, "How to do it?"

update: I feel what he meant by TDD test is feature testing.

3
  • 1
    Yeah, I know unit test != TDD. My question is about "TDD Tests" - a phrase the author of the article coined. (You would have to read the article to follow) He suggests that writing "TDD Tests" is not the same as writing unit tests - as if they were a separate category of test. My question is whether this is true or not.
    – asgeo1
    Commented Jun 24, 2011 at 5:35
  • 3
    -1 for not answering the question
    – David
    Commented Jun 24, 2011 at 6:19
  • 1+ for noticed asgeo1, David.
    – Armando
    Commented Jun 24, 2011 at 15:20
0

I don't think there's really a single correct answer to this. TDD gives some guidance what sort of tests you should write, and how they should relate to the rest of your development.

Depending on who uses the term, however, "unit test" can cover such a wide range of possibilities that it almost inevitably covers most TDD tests -- but then, you'd probably have a hard time finding any test that at least a few people wouldn't be willing to call a unit test...

Bottom line: the tests advocated by TDD cover a fairly wide range, but there's at least some agreement about what they are. "Unit test" is so poorly defined that it can refer to almost any sort of test. As a term, it means almost nothing.

0

There used to be an IEEE definition for unit testing that I liked because it was more general and didn't go with the whole TDD concept of testing the smallest testable units possible. Now that definition has been altered to match the EP-version which I find silly. To me, unit testing means checking points of interaction between larger sets of components where you're most likely to have some volatility when changes are made. If you see variations on definitions out there, it's because the phrase got co-opted by the EP/TDD crowd. I find it one of the more irritating phenomena in programmer culture. Why can't we just come up with new phrases rather than re-invent old ones?

Not the answer you're looking for? Browse other questions tagged or ask your own question.