This is essentially a question about writing clean tests
To attempt to answer the question, I will use as reference the book Clean Code: A Handbook of Agile Software Craftsmanship
The Chapter 9 of the book talks about Unit Tests and addresses most (if not all) of the questions that were asked.
WHAT utility classes are is a Domain-Specific Testing Language
This is defined as such:
(...) Rather than using the APIs that programmers use to manipulate the system, we build up a set of functions and utilities that make use of those APIs and that
make the tests more convenient to write and easier to read. These functions and utilities
become a specialized API used by the tests. They are a testing language that programmers use to help themselves to write their tests and to help those who must read those
tests later on.
This is exactly what is being asked. The author considers that writing Domain-Specific Testing language is not only an option but also a step toward improving test quality.
WHY one should care about writing those utility classes?
(...) tests must change as the production code
evolves. The dirtier the tests, the harder they are to change. The more tangled the test code,
the more likely it is that you will spend more time cramming new tests into the suite than it
takes to write the new production code. As you modify the production code, old tests start
to fail, and the mess in the test code makes it hard to get those tests to pass again. So the
tests become viewed as an ever-increasing liability.
The author clearly states:
The moral of the story is simple: Test code is just as important as production code. It
is not a second-class citizen. It requires thought, design, and care. It must be kept as clean
as production code.
With those quotes, I think it is reasonable that the question
In TDD, is it bad practice to write helper class for unit tests? should be handled with just as much care as any code that would be written in production.
What is a clean test?
Three things. Readability, readability, and readability. Readability is perhaps even more important in unit tests than it is in production code. What
makes tests readable? The same thing that makes all code readable: clarity, simplicity,
and density of expression. In a test you want to say a lot with as few expressions as
With that, it is clear that if there is code in the tests that could have its readability improved, it should be. It will make the tests more readable and hence, improve the tests quality. If you are writing the same 10 line code to compare two lists in a specific way, it would be advisable to refactor those 10 lines in a separate method to increase test readability.
Utility classes improve readability.
WHEN should one care about writing those utility classes?
This testing API is not designed up front; rather it evolves from the continued refactoring of test code that has gotten too tainted by obfuscating detail.
You are not going to design it as part of a public API that needs to be checked by unit tests. This testing API will be designed as the tests grows and as part of the refactor of the tests.
As stated by @Doc Brown:
The TDD cycle ("red-green-refactor") is exactly about evolving components by continued refactoring instead of designing them up-front.
This means that writing the utility classes fall under the third step in the TDD cycle. The refactor step.
WHERE should you put your utility classes?
The X Unit Patterns also define the concept of Test Utility Method and it has the following text in its Implementation Details guidelines:
Writing the reusable Test Utility Method is pretty straight-forward. The bigger question is where we would put it. If the Test Utility Method is only needed in Test Methods in a single Testcase Class (page X) then we can put the Test Utility Method onto that class but if we need it from several classes, the solution is a bit more complicated. It all comes down to type visibility. The client classes need to be able to see the Test Utility Method and the Test Utility Method needs to be able to see all the types and classes it depends on. When it doesn't depend on many or when everything it depends on is visible from a single place, the Test Utility Method can be put into a common Testcase Superclass (page X) we define for our project or company. If it depends on types/classes that cannot be seen from a single place that all the clients can see, it may be necessary to put it on a Test Helper in the appropriate test package or subsystem. In larger systems with many groups of domain objects, it is common to have one Test Helper for each group (package) of related domain objects.
Should one Unit Test the utility classes?
This question has already been asked in different shapes, one being:
Unit testing utility classes
The most well voted answer clearly states:
(...) YES, write tests for utility methods. NO, don't try to decouple them from other tests. Simply assume that trivial utility functions work correctly, as verified by their own tests. Doing anything more is more effort for no gain whatsoever.
This is essentially saying that the functionality provided by the utility methods will be tested in conjunction with the unit tests that are being written for the target public API. If there is a bug in the utility method you are using in the test, it should appear as a failed test for the public API you are testing.
This could also be seen when asking the question: Should you unit test your unit tests? Of course not. The whole idea of TDD is writing simple unit tests to verify desired behavior of your target code with the premise that:
- The unit test code should be simple enough for it to be easy to write and verify
- If there is a bug in the test case, it should appear as a failing test against your business logic.
By using the same reasoning, the utility method should be simple enough for verification and be trusted. If there is a bug in it, it should show in the tests that are being written anyway.
The test utility method is a simplification of the tests you are writing and should be handled as such.
Additionally consider the discussion in the following question:
In TDD, should I add unit tests to refactored code?
As stated in many of the answers:
Programmer tests should be sensitive to behavior changes and insensitive to structure changes. -- Kent Beck, 2019
The utility method, being defined as a refactor of duplicated code in unit tests, are by definition not sensitive to behavior changes. Hence, it should not be tested.
Refactoring is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior.
As unit tests are written specifically to test the behavior, it doesn't make sense for you to require additional unit tests after refactoring.
Refactoring can inherently never lead to needing additional unit tests that were not needed before.
In TDD, is it bad practice to write helper class for unit tests?
No! Not at all! In fact, it is a good practice as it improve readability, the main quality that test code should have. Just keep in mind that before starting writing many helper classes that will only be used in one place, it is advisable to first have the unit tests working before refactoring duplicated code into helper classes.
Should testing helper classes be unit tested?
Not directly. Helper classes only should exist as product of refactoring of unit tests for readability. Hence, just as regular unit tests, they should not be unit tested. Their correct behavior will be indirectly tested by their coupling to the unit tests being written for the target public API. If something is wrong, the tests should fail.
Additionally, do not reinvent the wheel
Unit tests are supposed to be simple. Because of that, many times the language or the test framework being used provides a clean way of achieving the desired duplicated piece of code.
Because of that, prefer the already existing methods to implementing your own utility classes. Do not reinvent the wheel. This whole question about implementing and testing or not utility classes can often be entirely avoided.
For instance, the example of comparing if two collections have the same elements out of order could be solved by a method such as CollectionAssert.AreEquivalent(ICollection, ICollection). (thanks for @Greg Burghardt for pointing that in the comments of the question).
It is not a coincidence that the concept of a Domain-Specific Testing Language was defined. Those utility classes are often better justified when the handle the DOMAIN of the application, not general data structures. For that, there are usually standard implementations.