19

In researching unit testing best practices to help put together guidelines for my organization, I've run into the question of whether it is better or useful to separate test fixtures (test classes) or to keep all tests for a single class in one file.

Fwiw, I am referring to "unit tests" in the pure sense that they are white-box tests targeting a single class, one assertion per test, all dependencies mocked, etc.

An example scenario is a class (call it Document) that has two methods: CheckIn and CheckOut. Each method implements various rules, etc. that control their behavior. Following the one-assertion-per-test rule, I will have multiple tests for each method. I can either place all of the tests in a single DocumentTests class with names like CheckInShouldThrowExceptionWhenUserIsUnauthorized and CheckOutShouldThrowExceptionWhenUserIsUnauthorized.

Or, I could have two separate test classes: CheckInShould and CheckOutShould. In this case, my test names would be shortened but they'd be organized so all tests for a specific behavior (method) are together.

I'm sure there are pro's and con's to either approach and am wondering if anyone has gone the route with multiple files and, if so, why? Or, if you've opted for the single file approach, why do you feel it is better?

  • 2
    Your test method names are too long. Simplify them, even if it means a test method would contain multiple assertions. – Bernard Dec 13 '11 at 15:00
  • 12
    @Bernard: it is considered bad practice to have multiple assertions per method, long test method names are however NOT considered bad practice. We usually document what the method is testing in the name itself. E.g. constructor_nullSomeList(), setUserName_UserNameIsNotValid() etc... – c_maker Dec 13 '11 at 15:14
  • 4
    @Bernard: i use long test names (and only there!) too. I love them, cause it is so clear what was not working (if your names are good choices ;)). – Sebastian Bauer Feb 3 '12 at 5:26
  • Multiple assertions are not a bad practice, if they all test the same result. Eg. you wouldn't have testResponseContainsSuccessTrue(), testResponseContainsMyData()and testResponseStatusCodeIsOk(). You would have them in a single testResponse()which has three asserts: assertEquals(200, response.status), assertEquals({"data": "mydata"}, response.data) and assertEquals(true, response.success) – Juha Untinen Jun 25 at 10:15
17

It's rare, but sometimes it makes sense to have multiple test classes for a given class under test. Typically I would do this when different setups are required, and shared across a subset of the tests.

  • 3
    Good example why that approach was presented to me in the first place. For instance, when I want to test the CheckIn method, I always want the object under test to be setup one way but when I test the CheckOut method, I require a different setup. Good point. – SonOfPirate Dec 13 '11 at 15:32
14

Can't really see any compelling reason why you would split a test for a single class into multiple test-classes. Since the driving idea should be to maintain cohesion on a class-level, you should strive for it on a test-level as well. Just some random reasons:

  1. No need to duplicate (and maintain multiple versions of) set-up code
  2. Easier to run all tests for a class from an IDE if they are group in one test-class
  3. Easier troubleshooting with one-to-one mapping between test-classes and classes.
  • Good points. If the class under test is an awful kludge, I would not rule out several test classes, where some of them contain helper logic. I just do not like very rigid rules. Other than that, great points. – Job Dec 13 '11 at 15:10
  • 1
    I'd eliminate duplicate setup code by having a common base class for the test fixtures working against a single class. Not sure I agree with the other two points at all. – SonOfPirate Dec 13 '11 at 15:29
  • In JUnit e.g. you may want to test things using different Runners which leads to the need for different classes. – Joachim Nilsson Jan 18 '17 at 13:03
  • As I write a class with a large number of methods with complex math in each one, I find I have 85 tests and have only written tests for 24% of the methods so far. I find myself here because I was wondering if Its insane to fork this massive number of tests into separate categories somehow so I can run subsets. – Mooing Duck Oct 11 '17 at 12:22
7

If you are compelled to split unit tests for a class across multiple files, that may be an indication that the class itself is poorly designed. I can't think of any scenario where it would be more beneficial to split up the unit tests for a class that reasonably adheres to the Single Responsibility Principle and other programming best practices.

Furthermore, having longer method names in unit tests is acceptable, but if it bothers you, you can always rethink your unit test naming convention to shorten the names.

  • Alas, in the real world not all classes adhere to SRP et al... and this becomes an issue when you are unit testing legacy code. – Péter Török Dec 13 '11 at 15:12
  • 3
    Not sure how SRP relates here. I have multiple methods in my class and I need to write unit tests for all of the different requirements that drive their behavior. Is it better to have one test class with 50 test methods or 5 test classes with 10 tests each that relate to a specific method in the class under test? – SonOfPirate Dec 13 '11 at 15:30
  • 2
    @SonOfPirate: If you need so many tests, that might mean your methods are doing too much. So SRP is very much relevant here. If you apply SRP to classes as well as methods, you will have lots of small classes and methods that are easy to test and you will not need so many test methods. (But I agree with Péter Török that when you test legacy code, good practices are out the door and you just have to do the best you can with what you are given...) – c_maker Dec 13 '11 at 15:47
  • Best example I can give is the CheckIn method where we have business rules that determine who is allowed to perform the action (authorization), if the object is in the proper state to be checked in such as if it is checked-out and if changes have been made. We will have unit tests that verify the authorization rules are being enforced as well as tests to verify that the method behaves correctly if CheckIn is called when the object is not checked-out, not changed, etc. That is why we end up with many tests. Are you suggesting this is wrong? – SonOfPirate Dec 13 '11 at 16:12
  • I don't think there are hard-and-fast, right-or-wrong answers on this subject. If what you're doing is working for you, that's great. This is just my perspective on a general guideline. – FishBasketGordo Dec 13 '11 at 16:30
2

One of my arguments against separating the tests into multiple classes is that it becomes harder for other developers on the team (especially those who aren't as test savvy) to locate the existing tests (Gee I wonder if there is already a test for this method? I wonder where it would be?) and also where to put new tests (I'm going to write a test for this method in this class, but not quite sure if I should put them in a new file, or an existing one?)

In the case where different tests require drastically different setups, I have seen some practitioners of TDD put the "fixture" or the "setup" in different classes/files, but not the tests themselves.

1

I wish I could remember/find the link where the technique I've chosen to adopt was first demonstrated. Essentially I create a single, abstract class for each class under test that contains nested test fixtures (classes) for each member being tested. This provides the separation originally desired but keeps all of the tests in the same file for each location. In addition, this generates a class name that allows easy grouping and sorting in the test runner.

Here is an example how this approach is applied to my original scenario:

public abstract class DocumentTests : TestBase
{
    [TestClass()]
    public sealed class CheckInShould : DocumentTests
    {
        [TestMethod()]
        public void ThrowExceptionWhenUserIsNotAuthorized()
        {
        }
    }

    [TestClass()]
    public sealed class CheckOutShould : DocumentTests
    {
        [TestMethod()]
        public void ThrowExceptionWhenUserIsNotAuthorized()
        {
        }
    }
}

This results in the following tests appearing in the test list:

DocumentTests+CheckInShould.ThrowExceptionWhenUserIsNotAuthorized
DocumentTests+CheckOutShould.ThrowExceptionWhenUserIsNotAuthorized

As more tests are added, they are easily grouped and sorted by class name which also keeps all tests for the class under test listed together.

Another advantage of this approach that I've learned to leverage is the object-oriented nature of this structure means I can define code inside the startup and cleanup methods of the DocumentTests base class that will be shared by all nested classes as well as inside each nested class for the tests it contains.

1

Try to think the other way around for a minute.

Why would you have only one test class per class ? Are you doing a class test or a unit test ? Do you even depend on classes ?

A unit test is supposed to be testing a specific behaviour, in a specific context. The fact that your class already has a CheckIn means there should be a behaviour that needs it in the first place.

How would you think about this pseudo-code :

// check_in_test.file

class CheckInTest extends TestCase {
        /** @test */
        public function unauthorized_users_cannot_check_in() {
                $this->expectException();

                $document = new Document($unauthorizedUser);

                $document->checkIn();
        }
}

Now you're not directly testing the checkIn method. You're testing a behaviour instead (which is to check in fortunately ;)), and if you need to refactor Document and split it into different classes, or merge another class into it, your test files are still coherent, they still make sense since you're never changing the logic when refactoring, only the structure.

One test file for one class simply makes it harder to refactor whenever you need it, and tests also make less sense in term of domain/logic of the code itself.

0

In general I would recommend thinking about tests as testing a behaviour of the class rather than a method. I.e. some of your tests may need to call both methods on the class in order to test some expected behaviour.

Usually I start out with one unit test class per production class, but may eventually split that unit test class into several test classes based on the behaviour they tests. In other words I would recommend against splitting the test class into CheckInShould and CheckOutShould, but rather splitting by behaviour of the unit under test.

0

1. Consider what is likely to go wrong

During initial development, you know fairly well what you are doing and both solutions will probably work OK.

It gets more interesting when a test fails after a change much later. There are two possibilities:

  1. You have seriously broken CheckIn (or CheckOut). Again, both the single-file as well as the two-file solution are OK in that case.
  2. You have modified both CheckIn and CheckOut (and perhaps their tests) in a way that makes sense for each of them, but not for both of them together. You have broken the coherence of the pair. In that case, splitting the tests over two files will make it harder to understand the problem.

2. Consider what tests are used for

Tests serve two main purposes:

  1. Automatically check the program still works correctly. Whether the tests are in one file or several does not matter for this purpose.
  2. Help a human reader make sense of the program. This will be much easier if tests for closely related functionality are kept together, because seeing their coherence is a rapid road to understanding.

So?

Both these perspectives suggest keeping tests together can help, but won't hurt.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.