3

When I am using the classical style of unit testing, how do I keep the number of test cases for an object that has many collaborators from growing too large? And how do I keep the setup of each test case from growing too large?

In Unit Testing Principles, Practises, and Patterns, Vladimir Khorikov recommends I use the classical style of unit testing, which means testing "units" of behaviour and not using test doubles to isolate the object that I am testing from other objects that it depends on.

When I do not use test doubles to isolate my objects from each other, it seems an object near the top of my object graph that is responsible for coordinating several other objects in order to validate input, persist some data, and send an email requires many unit tests, each with a lengthy test setup, because there are many possible combinations of input and thus many possible outcomes that must be tested.

I have difficulty understanding and maintaining the tests of an object with a large number of unit tests, each of which differs only slightly from the other test cases in the same file.

However, getting rid of one or more test case creates the possibility that the object I am testing has a bug that is only discoverable when testing that scenario.

The classical school states that it’s unit tests that need to be isolated from each other, not units. Also, a unit under test is a unit of behavior, not unit of code. Only shared dependencies should be replaced with test doubles. Shared dependencies are dependencies that provide means for tests to affect each other’s execution flow.

https://freecontent.manning.com/what-is-a-unit-test-part-2-classical-vs-london-schools/

3
  • By "too large" a number of test cases I mean 5000 lines of code to test a class and 300 test cases.
    – dbird
    Feb 11 at 20:16
  • 4
    Tests are code. While there has been a recent trend away from making test code "DRY", if there is a lot of shared code, I would argue in favor of refactoring it to provide a sane API for test setup. If the class does so much that it's hard to test all combinations of functionality, that also strikes me as something that should probably be refactored into more discrete units of functionality. Finally, I would deviate from the "don't use test doubles" advice. IMO if two components are interacting and you're not using a double for one of them, this seems to me like an integration test.
    – user229044
    Feb 11 at 20:46
  • "5000 lines of codeand 300 test cases to test a class" can be perfectly fine, or not - this depends heavily on the size and complexity of the class under test. When the class itself has 150 member functions, 300 test cases are just 2 per member on average, which is not many. Of course, you can question the size of the class itself.
    – Doc Brown
    Mar 15 at 6:07

2 Answers 2

3

Pay the technical debt back

By "too large a number" I mean 5000+ lines of code to test one class

[...]

how do I keep the number of test cases for an object with many collaborators from growing too large?

This will sound obvious, but worth saying. divide and conquer. Get rid of the God Object antipattern first. Otherwise, you won't be addressing the real problem. A piece of source code with 5000 LoC is claiming attention and refactoring.

See if you can group up collaborations around behaviours or reasons to collaborate so you can build abstractions around them. Cut down GodObject's complexity ASAP.

Don't repeat yourself

And how do I keep the setup of each test case from growing too large?

Make reusable test code the same way you do with production code. Some things you can do

Utils & Static Factory Methods

These classes have methods with very concrete names recalling the ubiquitous language. Names tell what they populate or do (usually, prepare an object graph in a specific state and ready for a limited number of execution paths).

Complex graphs are built by calling several factory methods, each of which is simpler than the one orchestrating the calls.

Extends the test framework

How does the framework works? Does it allow you to implement custom assertions and predicates? If yes, craft custom ones around the domain's ubiquitous language.

public class MyGodObjectTest {

    @Test
    public void given_state_S_when_X_then_Y() {
      Object value = MyGodObjectDSL.createGodObjectS().doX();
      assertThat(value, eq(y));
    }

    @Test
    public void given_state_S_when_Z_then_W() {
      MyGodObject god = MyGodObjectDSL.createGodObjectS();
      god.doZ(input);
      /*
      * isInStateW():
      * - will do all the assertions involving state W. 
      * - can replace duplicated validations all over the test code.
      * - can be parametrized to make it even more reusable
      */
      assertThat(god, isInStateW());
    }

    @Test
    public void given_state_S_when_Y_then_failure_due_to_MyException() {
      MyGodObject god = MyGodObjectDSL.createGodObjectS();
      // Example of custom assertion
      assertThrows(() -> god.doY(input))
        .logThrowable(LOGGER)
        .isInstanceOf(MyException.class)
        .hasErrorCode("Code")
        .hasErrorCode("Message");
    }
}

The goal is to keep test methods clean! Get rid of repeated code. DRY applies to test code too.

Rearrange the test code

then, each test case modifies the setup, meaning that I cannot understand all of the inputs to a single test case just by reading the test case itself

Instead of 300 unit tests in a single class, make different test suites. Each test suite focuses on specific states of the object, state transitions or execution paths.

Say the number of sad execution paths is significantly higher than happy ones. In that case, you could make one test suite solely for sad paths and another for happy ones.

Add some literature

The tests for this class are difficult to maintain because it takes me hours to read and understand what each of the test cases is testing

Naming things is hard. Right?

Allow yourself to break the naming conventions of your programming language and give test methods names some literature. Your attention should be on the given,when and then, not on everything that comes before. How to prepare GodObject for the test is useless and irrelevant code unless it's part of the test case itself.

public void testSomethingNotDocumentedSoHopeForTheBest(){ ... }

// vs
/**
* MyGodObject goes into state "S" when ...
* In such a state, consumers are allowed to ... only.
* Unallowed actions are expected to break the execution and result in a {@link MyException}
*/
public class MyGodObjectStateSTest
  public void given_state_S_when_X_then_Y(){ ... }
  public void given_state_S_when_X_and_Z_then_Y(){ ... }
  public void given_state_S_when_X_but_no_Z_then_W(){ ... }
}

The "literature" can also ease finding code by IDE's lookups.

0

You don't really specify what your particular problem is. What is "too large" a number? Why do you find them hard to maintain?

I think my general advice would be.

  1. You can never have too many tests
  2. Make sure your tests actually test the product works

When people advocate for these "classical" style of almost integration testing they are pushing back against what can be worthless unit tests. Tests which test a bit of logic in isolation are fine as far as they go, but don't actually give you assurance that your product works.

You can write millions of these tests, and they are totally worth writing in most cases. But they are not your critical DOES THIS WORK Y/N tests and at the end of the day you need to have those style of tests for all your use cases.

4
  • By "too large a number" I mean 5000+ lines of code to test one class. The tests for this class are difficult to maintain because it takes me hours to read and really understand what each of the test cases are testing. Also because the amount of input for the object being tested is so large that the "arrange" step is done at the top of the file (so as not to have 100+ lines of setup in every test case) and then each test case makes a modification to the setup, meaning that I cannot read understand all of the inputs to a single test case just by reading the test case itself.
    – dbird
    Feb 11 at 20:10
  • 1
    break the setup into individual functions With10PecentOff() etc
    – Ewan
    Feb 12 at 11:19
  • 1
    tbh though it doesn't sound like the problem is classic vs london, its something specific to your application. And you dont really give any details
    – Ewan
    Feb 12 at 11:21
  • 1
    @dbird: This sounds more like you've underdeveloped/underdocumented your testing suite. Clearly, you have a higher need for complex testing - the test suite should reflect that. Just some ideas: you can outsource the arrange logic to a config file and/or develop a fixture that allows you to load and adjust a scenario using minimal syntax (e.g. instead of needing to adjust several data points, combine them in an easy to read method (e.g. fixture.WithPerson("John", "Doe", out var johnDoeId)). Overall, I suspect you're mashing too much things in a single test file. Just separate it.
    – Flater
    Mar 14 at 0:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.