We're looking to concentrate more on testing behaviour (as opposed to every class/method individually) in all our automated tests, as espoused by various high-profile online discussions recently. I like the idea of thinking about only what goes in and out of certain logical well-defined boundaries between components, but I'm having a hard time imagining how that relates to quite low-level detail of behaviour, that is only a small part of the overall requirement but still important to be correct.
As an example:
We have a component which accepts a customer order and takes some kind of significant action with it, let's say accepts it and starts processing manufacture. The main thrust of the requirement is about accepting and manufacturing, but there are many small things which need to be correct, and which if incorrect should cause an exception, for example that the order date needs to be valid. If we assume that I don't implicitly trust whatever date validation routine is being used, then there are loads of things I need to test there: basic format, leap years, not in the future, etc etc. In "non-behavioural" TDD I would definitely test those at the class/method level, and then never worry in my other tests about whether they'll work. But if I'm only testing behaviour, do I need to take all those success/fail cases and test them on the component boundary, for every situation where I'm interested in the date?
I can see a few options:
- Massively tedious repetition of millions of tests, as on the low-level methods, but higher up.
- As above, but condense "is the date valid" down into a separate method which I reuse in my behaviour tests, so although I do check that the date is valid, it's only one line per test. This is much more likely than the first option...
- Test only broad behaviour higher-up, and do have a low level set of tests just to ensure the validation routines work. This seems sensible, but the importance of behaviour tests not relying on implementation details has been stressed - the actual validation routine being used is an implementation detail, invisible at the component boundary, so even if I test that routine, can I officially then not test it elsewhere as well somehow?
I'm leaning towards the middle option, but I'm quite new to this, and would be interested what you think about it!
Higher-level tests also exercise the interactions between components. Were your premise true, we could simply dispense with unit tests altogether and just use higher-level tests.
I think that's true (if taken to extremes), and there are people who would try to achieve exactly that. Understanding the reasons not to do that are partly the driver for my question.
I think potentially the difference between certain situations is to what extent these low-level behaviours are perceptible at the component boundary - in the case of date validation they are, I can feed lots of different values that I know should be valid or invalid in different ways into the interface and see differences in response; in other cases the unit in question does its work invisibly, at least from a behaviour perspective (I could test loads of stuff in a unit test, but perhaps the way it's being used in the larger component being tested doesn't give it much scope for variation).
Repetition vs Refactoring
The main reason I included the hyperbolic "millions of tests" option is another (unstated) principle that I also understand is good to aim for: "don't do too much in any one test, test one thing only". If you're also trying to follow that as much as possible, and think of the various validation failure cases as separate "things", it seems to conflict a bit with normal DRY/refactoring approaches - you can't squish the validation into one method and use it over and over, because then when it goes wrong you haven't got such specific feedback about what to fix (although you do at least have a valuable works/doesn't work response).
If this bothers you, follow the practice of TDD'ers everywhere, and only test public methods. Naturally, this doesn't preclude you making your "is the date valid" method a public method
I don't think I like the sounds of making something arbitrarily public (without changing anything else) just so I can test it - I'm happy enough changing my APIs when I need to change them, rather than so they fit my testing approach. Maybe bundling it off into another class (with public methods) if it might be useful, that makes more sense to me (and I've heard people suggest exactly that in TDD conversations). But the question is not really concerned with getting access to the low-level detail, it's about whether testing at that level at all is a good idea.
At the end of the day, common sense rules. If a piece of internal functionality is complex enough that you believe it needs to be covered by its own tests, then by all means, write tests for it. Conversely, I seldom write tests for trivial getter/setter methods, or methods that are prima facie correct.
No argument from me about that! Depsite what else I've written I'm sure we'll end up with a pragmatic mixture of approaches that we've chosen to give us the confidence in our code we need. This is helping me decide what that might be!
So far I'm erring towards accepting slightly coarser tests at the API level (for e.g. things like our date validation example), while also setting up comprehensive unit tests for certain underlying classes - that way you get the full behaviour coverage, and nothing can slip through the net at the level which truly matters, but also you can have good confidence in and ease of maintenance for the complex low-level units.
