Should unit tests only cover 'functional' software

Question

We are using StructureMap in a new software development project. One of the team members has implemented a unit test that basically tests the StructureMap container configuration. It does this by doing the following;

• Counts number of instances of assemblies that are configured for classes in our application namespace.
• Defines expected instances at the class level
• Asserts that expected instances match total found instances.
• Asserts that expected instances match those defined in the test

An example of this is;

var repositories = container.GetAllInstances<IEnvironmentRepository>();
Assert.AreEqual(1, repositories .Count());
foundInstances = foundInstances + repositories .Count();

We also have 'unit tests' for the following class;

public MyClass(IEnvironmentRepository environmentRepository)
        {

        }

In these tests, we mock IEnvironmentRepository, so would not be injecting it from the container as would happen in the live system.

A colleague ignored the unit test on the structuremap config with a comment along the line of "Unit test only tests it's own configuration". This was obviously the purpose of the test and in my opinion is perfectly valid. I asked the guy who ignored the test to remove the structuremap configuration for IEnvironmentRepository (with the test still ignored) and run the full unit test suite, they all passed. We then ran the application and it fell over because the container configuration was now invalid. In my opinion, this proved the value of the test, my colleague still disagreed. He simply stated that we should not be testing configuration, but I deem this to be well within the remit of a unit test.

So a number of questions;

• Is it a valid unit test - We are testing the configuration of our container, not that structuremap works (but I can see the overlap)
• If not, how can you validate the configuration without testing it. How can you stop someone accidentally deleting a required line of code and checking it in?
• Should the MyClass unit test resolve the instance of IEnvironmentRepository from the container and pass this in?

Answer 1

This is a perfectly valid automated test to have. I call them "architecture tests" as they verify the soundness of your code base's skeletal components.

Is the IoC container able to resolve and compose all the object trees in the application? Can the auto Mapper map between all its registered objects without failing? Does the central layer in an Onion Architecture not reference anything external?

These tests can save you a lot of time when a configuration bug sneaks in, by pointing at the exact culprit. Good frameworks will give you very precise error messages about what's wrong and you get them as soon as you run the tests (ideally, continuously) instead of buried deep down a runtime stack trace if you're lucky.

Whether they are unit tests... probably not, but they still operate in memory for the most part and run pretty fast. Then again, I don't know, it's not like there was a universally accepted definition of unit test.

Answer 2

The problem with a test like this that test the internals of the program, rather than a requirement of it. Is that the test can fail even if the program works as required.

In your case, whenever you change the container setup, maybe you have a new dependency that needs injecting, you break your test.

Additionally, if you add the extra dependency requirement, but forget to add it to the container and change the container test. everything will pass, but your program will crash.

A better automated test would be to start the program up and see if it crashes.

You should catch these types of error at integration or UI testing even if they fall through the unit tests.

Having said that, the growing complexity of container setup is a pain in the arse. Perhaps some 'bad' tests are worth it.

Answer 3

Unit tests test code. Anything outside of this is "other" automated testing - call it what you will. You seem to be testing configuration here. If the configuration could change depending on the environment, it doesn't belong in a unit test. Consider adding a test attribute to indicate that the test is of a different type to the other tests.

Answer 4

UnitTest verify desired behavior of a unit in separation.

This means any kind of configuration is not in the scope of UnitTests.

Nonetheless you should have automated tests for your configurations, but these are not UnitTests...

Answer 5

Responsibility of dependency injection container is to glue different modules in one working application.

If you write automated tests for your application - you should have few" integration(or acceptence) tests which execute tests "from end to end", which will test whole pipeline of your application, that all modules involved in particular test case are glued together correctly.

So those integration tests will fail if dependency injection container was not configured properly. Which make unit tests for container itself useless, because integration test should show possible errors in container configuration.

You don't need to cover all possible test cases in integration tests, just one test case per feature which cover full way from UI to database.

If integration test cases do not cover instantiation of some particular dependency - you simply add such one.

With integration tests you can freely change containers without re-writing unit tests for their configuration.

Answer 6

IMO, the answers are:

1. Is it a valid unit test - We are testing the configuration of our container, not that structuremap works (but I can see the overlap)

   • It's valid unit test for structuremap, not for your project, because a unitary test tests some specific code, mocking all dependencies if necessary in order to test the logic implemented. The config logic is implemented inside structuremap, therefore this library must be well tested and must contain unit tests like this one you mentioned, and more: it should contain hundreds of tests like this, dynamically mocking several configs at runtime and testing to see if the container behaves as it should.

2. If not, how can you validate the configuration without testing it. How can you stop someone accidentally deleting a required line of code and checking it in?

   • You can test the config manually in the environment needed, and also you can create an automation for this (automated test), that tests for the specific configuration that you need (no need to mock stuff at runtime).

3. Should the MyClass unit test resolve the instance of IEnvironmentRepository from the container and pass this in?

   • No, this is a perfect unit test, because you mock the dependency and test MyClass logic in an isolated way.