9

We are using StructureMap in a new software development project. One of the team members has implemented a unit test that basically tests the StructureMap container configuration. It does this by doing the following;

  • Counts number of instances of assemblies that are configured for classes in our application namespace.
  • Defines expected instances at the class level
  • Asserts that expected instances match total found instances.
  • Asserts that expected instances match those defined in the test

An example of this is;

var repositories = container.GetAllInstances<IEnvironmentRepository>();
Assert.AreEqual(1, repositories .Count());
foundInstances = foundInstances + repositories .Count();

We also have 'unit tests' for the following class;

public MyClass(IEnvironmentRepository environmentRepository)
        {

        }

In these tests, we mock IEnvironmentRepository, so would not be injecting it from the container as would happen in the live system.

A colleague ignored the unit test on the structuremap config with a comment along the line of "Unit test only tests it's own configuration". This was obviously the purpose of the test and in my opinion is perfectly valid. I asked the guy who ignored the test to remove the structuremap configuration for IEnvironmentRepository (with the test still ignored) and run the full unit test suite, they all passed. We then ran the application and it fell over because the container configuration was now invalid. In my opinion, this proved the value of the test, my colleague still disagreed. He simply stated that we should not be testing configuration, but I deem this to be well within the remit of a unit test.

So a number of questions;

  • Is it a valid unit test - We are testing the configuration of our container, not that structuremap works (but I can see the overlap)
  • If not, how can you validate the configuration without testing it. How can you stop someone accidentally deleting a required line of code and checking it in?
  • Should the MyClass unit test resolve the instance of IEnvironmentRepository from the container and pass this in?
  • 10
    9 out of 10 disagreements on tests arise from the fact that frameworks support automated tests in all their forms, and people want to get into semantics of whether a particular automated test is a good and proper unit test or not. The test you describe sounds like the kind of not-quite-unit-test-test that may very well be useful to have and automate (and run on checkin) -- just don't call it a unit test. Ask if your colleague would sleep better at night if the test lived in its own feature/folder that was clearly separated. – Jeroen Mostert Aug 21 '17 at 12:28
  • 2
    That's my opinion as well, probably useful, and while not strictly a unit test, it does add value and this has been proven. His response was that the other unit tests would have picked this up, but in my opinion, if they were written as strict unit tests, you would be mocked the dependencies and therefore would never know if the configuration was valid until you used it. – ChrisBint Aug 21 '17 at 12:31
  • 4
    Your colleague has a point when he says not to test configuration, inasmuch as genuine configuration that can actually vary per deployment can't/shouldn't be tested -- who's to say "red" is wrong and "blue" isn't? The test would be tightly coupled to one setup. But configuration that is tied to code artifacts is a bit of an exception, because it doesn't vary and there are clearly ways to get it wrong. Ideally, you'd have such configuration generated at build time from DRY metadata, but where this isn't feasible a test like this does add value. Better that than an avoidable deployment error. – Jeroen Mostert Aug 21 '17 at 12:39
  • 2
    What you're describing does not test a unit, it tests the configuration of a third party piece of software. It's fantastically useful to have tests which test these things, but they are integration tests, not unit tests, and the disconnect there may be the root of the disagreement. – Phoshi Aug 21 '17 at 14:19
  • 3
    @ChrisBint Goodness gracious no, I've written a bunch of container tests myself. They have a lot of value, they just aren't unit tests. That's fine, integration tests are extremely valuable for catching things unit tests can't. – Phoshi Aug 21 '17 at 16:04
13

This is a perfectly valid automated test to have. I call them "architecture tests" as they verify the soundness of your code base's skeletal components.

Is the IoC container able to resolve and compose all the object trees in the application? Can the auto Mapper map between all its registered objects without failing? Does the central layer in an Onion Architecture not reference anything external?

These tests can save you a lot of time when a configuration bug sneaks in, by pointing at the exact culprit. Good frameworks will give you very precise error messages about what's wrong and you get them as soon as you run the tests (ideally, continuously) instead of buried deep down a runtime stack trace if you're lucky.

Whether they are unit tests... probably not, but they still operate in memory for the most part and run pretty fast. Then again, I don't know, it's not like there was a universally accepted definition of unit test.

  • Ironically, this is pretty much how I explained it to my colleague and even with the validation (delete one of the container instances and run the application) he still was not seeing any value. I understand everyone has their own opinion, and I voiced mine ;) I love the term "architecture test", I am going to steal that! – ChrisBint Aug 21 '17 at 18:38
6

The problem with a test like this that test the internals of the program, rather than a requirement of it. Is that the test can fail even if the program works as required.

In your case, whenever you change the container setup, maybe you have a new dependency that needs injecting, you break your test.

Additionally, if you add the extra dependency requirement, but forget to add it to the container and change the container test. everything will pass, but your program will crash.

A better automated test would be to start the program up and see if it crashes.

You should catch these types of error at integration or UI testing even if they fall through the unit tests.

Having said that, the growing complexity of container setup is a pain in the arse. Perhaps some 'bad' tests are worth it.

1

Unit tests test code. Anything outside of this is "other" automated testing - call it what you will. You seem to be testing configuration here. If the configuration could change depending on the environment, it doesn't belong in a unit test. Consider adding a test attribute to indicate that the test is of a different type to the other tests.

  • Configuration is static, it is not driven by environment, all classes that exist in the configuration will be used on all environments in the same way. Yes, the number of instances that could be in the config should match the number of instances in the config, that is part of the test. As my example showed, removing IEnvironmentRepository allowed the other unit tests to pass. The specific container test would have failed on 2 Asserts; 1 - The total number of possible instance declarations did not match, and 2 - the specific number of instannces of IEnvironmentRepository would not match. – ChrisBint Aug 21 '17 at 13:01
  • 1
    The correctness of the container is defined by the coder. The fact that both the code under test and the test itself have to change for each modification immediately sets alarm bells ringing. DI is a means to an end and not the end in itself. It is perfectly possible to write code in a DI style without structuremap ergo it isn't a bona fide unit test in my view. The container of course needs to be proven but the efficacy of doing so with automated tests would seem to be somewhat moot with the limited information provided here. – Robbie Dee Aug 21 '17 at 13:47
  • 2
    Unit tests took 10 minutes to knock up. Deployment could take over an hour. – ChrisBint Aug 21 '17 at 14:26
  • 1
    Given part of the unit test validates specifically the existence of a single line in the configuration, not sure how that could not have been more isolated. The general count I could agree with. – ChrisBint Aug 21 '17 at 14:35
  • 1
    There could be some mileage in them then - a judgement call for you really. But they should be separated off either physically or via some attribute. – Robbie Dee Aug 21 '17 at 15:07
0

Responsibility of dependency injection container is to glue different modules in one working application.

If you write automated tests for your application - you should have few" integration(or acceptence) tests which execute tests "from end to end", which will test whole pipeline of your application, that all modules involved in particular test case are glued together correctly.

So those integration tests will fail if dependency injection container was not configured properly. Which make unit tests for container itself useless, because integration test should show possible errors in container configuration.

You don't need to cover all possible test cases in integration tests, just one test case per feature which cover full way from UI to database.

If integration test cases do not cover instantiation of some particular dependency - you simply add such one.

With integration tests you can freely change containers without re-writing unit tests for their configuration.

0

IMO, the answers are:

  1. Is it a valid unit test - We are testing the configuration of our container, not that structuremap works (but I can see the overlap)

    • It's valid unit test for structuremap, not for your project, because a unitary test tests some specific code, mocking all dependencies if necessary in order to test the logic implemented. The config logic is implemented inside structuremap, therefore this library must be well tested and must contain unit tests like this one you mentioned, and more: it should contain hundreds of tests like this, dynamically mocking several configs at runtime and testing to see if the container behaves as it should.
  2. If not, how can you validate the configuration without testing it. How can you stop someone accidentally deleting a required line of code and checking it in?

    • You can test the config manually in the environment needed, and also you can create an automation for this (automated test), that tests for the specific configuration that you need (no need to mock stuff at runtime).
  3. Should the MyClass unit test resolve the instance of IEnvironmentRepository from the container and pass this in?

    • No, this is a perfect unit test, because you mock the dependency and test MyClass logic in an isolated way.
-1

UnitTest verify desired behavior of a unit in separation.

This means any kind of configuration is not in the scope of UnitTests.

Nonetheless you should have automated tests for your configurations, but these are not UnitTests...

  • Where are you getting the definition of Units? – ChrisBint Aug 21 '17 at 14:35
  • I like the one of Roy Osherove in The Art Of Unittesting: A Unit is any chunk of (production-) code that has the same reason to change. I my world this usually ranges from a single class up to three or five... – Timothy Truckle Aug 21 '17 at 14:40
  • This is production code that is being tested. – ChrisBint Aug 21 '17 at 14:44
  • Just wanted to distract the nitpickers, not expecting it works the other way around too... ;o) – Timothy Truckle Aug 21 '17 at 14:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.