State Change Tests

Question

In Chapter 3 of his book The Art of Unit Testing: with Examples in C#, Roy Osherove describes the concept of testing state change of a system.

The example code under test he uses looks like this:

public class LogAnalyzer
{
    public bool WasLastFileNameValid { get; set; }

    public bool IsValidLogFileName(string filename)
    {
        WasLastFileNameValid = false;

        if (string.IsNullOrEmpty(filename))
        {
            throw new ArgumentException("filename has to be provided");
        }

        if (filename.EndsWith(".SLF", StringComparison.InvariantCultureIgnoreCase))
        {
            WasLastFileNameValid = true;
            return true;
        }
        return false;
    }
}

and we want to test state of the WasLastFileNameValid property.

To this end, the author uses the following test:

[Test]
public void IsValidFileName_WhenCalled_ChangesWasLastFileNameValid()
{
  LogAnalyzer la = MakeAnalyzer();

  la.IsValidLogFileName("badname.foo");

  Assert.False(la.WasLastFileNameValid);
}

However, I see the following issues with this test:

1. The 'outcome' part of the test name is ChangesWasLastFileNameValid, but the test doesn't really check whether the property value changes; it may have been false even before the call to IsValidLogFileName.
2. The test is only testing the one case where the last call was an invalid filename.

I would use the following test instead (using xunit.net)

    [Theory]
    [InlineData(true, "fileWithValidExtension.SLF", true)]
    [InlineData(true, "fileWithBadExtension.FOO", false)]
    [InlineData(false, "fileWithValidExtension.SLF", true)]
    [InlineData(false, "fileWithBadExtension.FOO", false)]
    public void IsValidLogFileName_WhenCalled_ChangesWasLastFileNameValid(
        bool preState, string filename, bool postState)
    {
        LogAnalyzer analyzer = new LogAnalyzer();
        analyzer.WasLastFileNameValid = preState;

        analyzer.IsValidLogFileName(filename);

        Assert.Equal<bool>(postState, analyzer.WasLastFileNameValid);
    }

Here I test whether the value changes, and I also test all scenarios. Is this a better test?

Answer 1

To answer your specific concerns:

1. Whether a state change occurs is not necessarily relevant, only that the correct state exists at the time of testing.

2. Both states should be tested against their relevant conditions, not just one or the other.

It's probably a better software design if the IsValidLogFileName() method is called at the moment that an evaluation of the file name is needed, rather than trying to maintain a state variable with that information. It should be redesigned to be stateless, in other words (all other things being equal).