1

I have a method GetReportAsync that takes one XML and generates another:

public async Task<string> GetReportAsync(string id)
{
    //  Get Order.xml from a file database
    var order = await _dataService.GetOrderAsync(id);

    var report = _reportGenerator.WriteReport(order);

    var xml = SerializeToXml(report);
    
    return xml;
}

As an automated testing tool, I write XMLUnit.NET snapshot tests that assert that an order will match a certain reference XML document, a stored snapshot.

[TestMethod]
[DataRow("12345")]
[DataRow("67890")]
public async Task Snapshots_Should_Match(string id)
{
    var actual = await _testClass.GetReportAsync(id);

    var expected = Input.FromFile($@"Snapshots\__snapshots__\{id}.xml");

    var diff = DiffBuilder
        .Compare(expected)
        .WithTest(actual)
        .Build();

    Assert.IsFalse(diff.HasDifferences());
}

Now, I realize that the Order.xml file can be retrieved in two ways:

  • Alternative A: Store a number of Order.xml files in the project and implement _dataService.GetOrderAsync to read from these files.
  • Alternative B: Get the actual Order.xml from a test (or production) database, just like in the real implementation.

Alternative A will assert that the method works given the order files in their exact state from when this snapshot test was written. However, I struggle to see the reason for such a guarantee.

Alternative B will give me tests that fail if the data service for some reason changes its response, possibly because of a non-backwards compatible change they introduce. To me this seems to give me more value than alternative A. However I do see that testing against a real database would possibly break some fundamental rules.

Furthermore, since the method is not writing to the database, then why not test against the production database instead of a test database?

4
  • 4
    ...then why not test against the production database instead of a test database? -- For several reasons. I think you can probably discard this as a possible option. If you want to exercise a real database, create one solely for testing purposes. Jan 4, 2021 at 15:33
  • @RobertHarvey Yes, I understand that, but what are your thoughts about alternative A and B? Jan 5, 2021 at 9:23
  • 2
    It's the classic tradeoff between unit tests (essentially alternative A) and integration tests (essentially alternative B). Jan 5, 2021 at 12:51
  • 1
    What are you testing here? DiffBuilder? GetReportAsync? Both? You can test DiffBuilder without data sources, just with stubbs.
    – Laiv
    Jan 8, 2021 at 13:08

1 Answer 1

0

My thought is you should use the alternative B.

As I can see if you have to change the code to run the alternative A, such behavior is not correct.

Also, alternative B seems mostly like a snapshot test because it preserves a system functionality.

1
  • This answer is not so much stating an incorrect observation but applying it overzealously. When taken to this degree, one could argue that a codebase that is currently without inversion of control should therefore not be changed in order to make itself testable. That's clearly not in the spirit of the given advice. What is correct, however, is that you shouldn't be amending your (already clean) code in order to make something testable that previously wasn't. The difference here is in what constitutes clean code and reasonable testability, which is a standard this answer does not establish.
    – Flater
    Feb 1, 2023 at 1:14

Not the answer you're looking for? Browse other questions tagged or ask your own question.