Unit tests: tree of dependent tests

Question

I frequently encounter situation where I have very basic tests, and if those fail, it doesn't make sense to run many other tests. Therefore I want to declare a tree of dependencies between my tests, where tests are only run if all pre-conditions tests are passing. two big advantages I can think of are:

1. During debugging I do not see more advanced tests failing, making the problem more obvious.
2. Tests will complete faster.

Example: I have implemented running average filter. First test is that the signal has the same amount of samples after applying this filter. Next, I want to test further mathematical properties of my signal, but tests are bound to fail if first condition is not satisfied (same amount of samples).

Is this a known concept? is this what test suites are? I am also interested if this is available in pytest which I currently use.

Answer 1

I see a couple of problems with this approach:

1. Unit tests are supposed to work independently of each other. Your proposal would couple them together.

2. Unit testing tools often use concurrency mechanisms and automation tools that ensure all of the tests complete on a regular basis and in a timely manner. These mechanisms would break under your scheme.

If there are prerequisites to running your suite of unit tests (such as "the code shall compile" or "the test database shall be set to baseline"), you should certainly establish that those prerequisites have been met before your suite of unit tests execute. But I don't think you need a UI test hierarchy to accomplish that.

If you're having difficulty getting all of the unit tests to run in a timely manner, consider standing up a CI server that runs your unit tests for you.

Answer 2

Most unit tests should execute very quickly, with tests taking ~1 second being slow. Unless you have a huge suite of tests, or your tests run slowly, executing all of the tests should not take a whole lot of time - maybe a minute or two. Therefore, executing tests that are going to fail is not that big of a problem. Name your tests smartly so that it's clear what is being tested, and tracking down the point of failure for multiple tests shouldn't be too hard.

If you have (a lot of) tests that run slowly, your test framework may support grouping those tests together and running them separately. NUnit has support for the Category attribute in C#, for instance, and you can run tests from a specific category. This may also integrate with your build server, so you can create a separate build configuration which runs only the slow tests, and is dependent on the build config which runs the fast tests passing. You may set the slow test config to only run once a day, for instance.

This could achieve what you're looking for, but before doing that I'd take a hard look at why your tests run so slowly. Unit tests are most beneficial when they easy to run and quick to run.