Testing early is important – but not everything has to be tested, and a lot of code is not served well by TDD-style unit tests.
- For a prototype, it's completely fine to temporarily ignore best practices, as long as you will throw that code away later.
- Some code is really plain yet so central to your application that everything will break if it contains a problem – you don't need explicit tests for that either.
- And some things are really difficult to test in isolation. Then don't waste your time writing unit tests. This is often the case for business logic, or GUIs.
So a big part of efficient testing is knowing what not to test.
Tests capture requirements
But automated test cases have an extremely important dual role: They are not just verification that your system meets its requirements, they are also a description of these requirements – a description that can be automatically verified. How are you currently tracking your requirements? How are you ensuring that your system satisfies these requirements? Having domain experts manually test everything is a waste of time.
Easy testing with BDD
Behaviour-Driven Development is the least bureaucratic and most effective method I know. BDD suggests a TDD-like workflow, but on the level of your problem domain rather than on the level of unit tests.
You talk with domain experts to understand the required behaviour of the system. You write this down in a plain-text format that everyone involved can understand. Work through concrete examples. Suggest some inputs to the system, and ask the domain experts how the system should react.
Organize these plain-text notes into a machine-readable format and write an interpreter for them. This interpreter runs each scenario and verifies the system's responses.
Use whatever format is convenient but still readable by the domain experts. I've used ad-hoc formats, YAML files, Cucumber feature files and so on. A developer-oriented project I'm maintaining uses Makefiles to describe each scenario.
It's OK if the test runner is unable to verify some requirements. You'll have to check those manually for now, but it's still good to keep all requirements in the same place.
Pick a scenario and use a TDD-loop to implement it. Or don't. Use whatever workflow works for you.
The power of this approach is that these natural-language scenarios are test suite and requirements document at the same time. And the test runner decouples the tests from your implementation choices. If you change how you satisfy these requirements then you don't have to throw away all your tests, because your requirements have stayed the same. Instead, you only need to update the test runner.
And thanks to the test vs. test runner split, most test cases will be fairly compact.
Early testing saves effort
The earlier you can find a problem, the cheaper it is to fix that problem:
Early in development your design is still in flux and can more easily accommodate the necessary changes.
You limit the possible damage from that defect, e.g. time and other resources wasted by users. Fresh bugs are also easier to debug because you're still immersed in the context of that code.
You make more efficient use of your subject matter experts. Getting next-day feedback from your expert: good. Getting same-minute feedback from your tests: much better.
Of course, tests can never replace experts: whereas tests can verify the system (check for conformance with known requirements), humans can use their judgement to validate the system (check that it actually serves the business needs).
With a test-first approach, an implementation defect will be flagged as soon as the code is written. It's not possible to be quicker than that. This minimizes the cost of defects.
Paradoxically, writing tests together with the code also reduces the cost of writing either:
- Writing tests involves design work.
When you already have a clear idea of what to do and how to do it, writing that code is much easier.
- Testability is an important but easily forgotten design constraint.
When you are writing code together with the tests, writing those tests is much easier.
Early testing is faster than writing code first and tests much later. But of course, if time to market is more important than minimizing total costs, then deferring some of the testing effort (= piling up technical debt) can be a legitimate decision.
Prototypes should be small, quick, and discarded.
Prototypes (spikes) work best when they are a just quick feasibility study. The result of a prototype is not some software, but the knowledge of whether a particular approach works.
You have a multi-month project with complex requirements. That is no longer a prototype. Let's call it an alpha-quality project instead. What is your plan to transform this alpha software into a usable product?
If you restart from scratch, how will you make sure that all requirements are correctly satisfied by the rewrite? How will you carry over all the little details that you figured out? But a rewrite would imply “throwing away” multiple months of work, so this is unlikely to happen.
If you will incrementally refactor your code, it was a mistake to treat it as a prototype. Well, it's in the past. But how will you make sure that any refactoring is safe and keeps satisfying the requirements? Tests would help with that.
So under the assumptions that you will need tests anyway and that writing tests together with the code is the cheapest way to write tests – yes, you might have made a mistake.
The good news is that now is still a good time to start writing some tests. Don't go overboard with this.
Don't write meticulous unit tests for every detail quite yet.
Don't consider code coverage metrics at this phase of the project.
But do start running high-level tests for any new feature or business rule change from now on.