I'm relatively new to TDD and have been thinking a lot about how to manage the perpetually growing pool of tests that comes with it.
One of my biggest concerns is about false positives.
In my experience, it's not uncommon at all for a feature to undergo fairly massive changes over time. To implement a change we would first write a test (or many tests) for the new behaviour and then code the change. Ideally, all the tests would pass once the feature change has been correctly implemented.
But what about the tests that covered the first incarnation of the feature? There is no guarantee that:
- they are still relevant;
- they still pass;
- their pass/fail status is even correct.
Reviewing/updating/deleting these tests in a relatively small pool of tests may be manageable, but what about when you have thousands or tens of thousands of tests? It seems to me that there is a big risk that your collection of tests will eventually contain many false positives. And, considering that these tests are a form of system documentation, this is pretty bad!
Question: How do you mitigate the risk of accumulating False Positive tests when applying TDD or BDD (or anything that eventually leads to having a massive collection of tests)?