Given a large group (50+) of programmers:

I have personally seen the wide spectrum tests that are possible for the same problem, even on the first test. So, if TDD is design, how do you know your TDD is optimal for the current problem, and how do you know it is not? Following the first test, is the approach for reviewing the consecutive tests any different, and if so, how?


TDD is not design - it is a design process.

A main artifact a TDD process eventually give you is the unit test suites, which should (to some degree) attest to the actual code's scope and capabilities.
Good test suites give some assurances that the code works, and added security that future code changes can be done without breaking past code, as failing tests will prevent broken code from leaking to the system.

So, if you want to evaluate a test suite written during TDD you should look for:

Tests should be easily read. It should be obvious what each test does.
A good test-suite describes the code functionality as a PRD (Product Requirement Documents) would, down to the edge-cases.
Tests that you can't understand what they do, are confusing, filled with 'magic numbers', long lines, illogical flow, vague objectives are bad. In a few weeks nobody will remember their purpose, and when they break, no one will no why or how to fix them, and they will be deleted from the code base, or worse yet - simply ignored, and left as broken tests in your code.

Individual tests should be short. They should test a single thing or use case.
When a good test fails, its name is sufficient to send the programmer to the correct class and method, maybe even line of code where he should look for the problem.
If a test is long, and has many moving parts - when it fails it just means now the programmer needs to first find out where the test failed, and the bug hunt, instead of being focused in the code, starts from combing the test itself.

Tests should be deterministic.
When a good test fails, it fails every time.
Tests which are time bound (rely on Time.now()), statistical (use Random), or rely on external assumption (that a specific row is in a table, that a web service is available and working...) are not reliable, since they might pass when they should fail, or fail when they should pass, and eventually be ignored by the team.
Make sure a test stubs out all external dependencies (yes, Time and Random as well).
You can't test randomness!
I once saw a test which tested that an array shuffle does not return a sorted array - well, guess what - the sorted array is a valid random result!

A good test suite is comprehensive.
When you read through a test suite, try to think of all the edge-cases, and see if they are tested.
Poor test suites fail to cover non-mainstream use-cases, and allow functional bugs to exist in a code which passes them.

A good test suite should be DRY.
This helps both readability and maintainability. As the code base evolves, so are the tests.
If changes to tests involve cumbersome search and replace, and hours of programmer time to make failing tests pass (by updating the tests) each time the requirements change, it will make the whole approach fail, as programmers will simply stop maintaining the tests, and abandon TDD altogether as "too time consuming".
Good tests, when DRY, are easier to maintain, as code changes should involve less changes in the tests, and hence less time spent in old test code.

A good test suite should not take more than a few seconds to run.
The longer a test suite takes to run, the less times it will be run.
I/O, especially networking, but also heavy disk reads/writes, complex DB queries, etc. should be faked or stubbed.
Never use sleep() in tests. Fake time instead.
Sometimes there are tests you have to do which take more time. Contain them in a special 'long running' section, perhaps in the "integration" test suite, and have them run less often (nightly perhaps).

Not the answer you're looking for? Browse other questions tagged or ask your own question.