Unit test structure focused on responsibilities

Question

I'm not going to ask what is the best unit test structure or how to do it again. There are already enough questions about it. Instead I'd like you to tell me what else I could consider in the structure that I came up with (borrowing from other ideas and adding new ones).

I've been thinking about how can I improve my test structure because after writing a few tests it was becoming harder and harder to see what cases I've already covered and which are still missing. A long list of test methods didn't really show what features they were actually testing. They were testing some cases but what kind of cases? They are mostly too unspecific to me. I wanted them to be a documentation of what is required for a class/method to work and how it works rather then what it does if it doesn't or does if it works.

So I flipped the logic and instead of writing what really happens I write what the class/method must recognize as an incorrect or correct usage.

To implement it I added two new layers to the test structure prerequisite tests and usage tests. This allows me to better keep in mind what I have already tested and what is missing. Let's try it on the following (simplified) example:

public class Configuration
{
     public static XDocument LoadConfiguration(string fileName)
     {
         if (string.IsNullOrEmpty(fileName)) throw new ArgumentNullException("fileName");

         if (!File.Exists(fileName)) throw new FileNotFoundException(fileName);

         return XDocument.Load(fileName);
     }
}

To test it I first define (if necessary) some tests for testing the prerequisites that must be satisfied before it is even possible to work with a class or method.

Then I test the different possible usages of the class or method.

[TestClass]
public class ConfigurationTests
{
    [TestClass]
    public class LoadConfiguration_TestPrerequisites
    {
        [TestMethod]
        public void FileNameMustNotBeNull()
        {
            // assert that ArgumentNullException is thrown when fileName is null
        }

        [TestMethod]
        public void FileNameMustExist()
        {
            // assert that FileNotFoundException is thrown when file doesn't exist
        }
    }

    [TestClass]
    public class LoadConfiguration_TestUsage
    {
        [TestMethod]
        public void MustImportValidXml()
        {
            // assert that configuration was loaded
        }

        [TestMethod]
        public void MustNotImportInvalidXml()
        {
            // assert that some other exception was thrown on invalid xml
        }
    }
}

My question is: What other key responsibilities I might be missing here besides prerequisites and usage? Somehow I'm not yet entirely happy with it and my gut tells me that I've overlooked something.

My goal is also to describe (name) the tests more naturally so they better show class/method's responsibilities and not what happens. I find it's easier to understand FileNameMustNotBeNull than Throws_FileNotFoundException. because the implementation may change (I could decide to return a null value if it the file name was null) but it still cannot be null.

Answer 1

Here are some broad categories of tests that might be what you need:

• handling invalid input (what you call prerequisites)

• correctness of happy-path logic (what you call usage) - this one may be large enough to be worth of decomposing further, but it's case-specific

• handling edge cases (where the input is at the edges of valid range)

• handling negative cases (testing that something undesired doesn't happen)

I wouldn't call them responsibilities though. A module should have only one responsibility (it's function).

Validation of prerequisites is not some 'secondary' responsibility of a module; it's only a way to ensure that the module is able to process the input (perform it's function) correctly.