I'm about to write a program and test(s) for it. This question is about how to structure the top-most level integration test.

At the top level, the main program takes a chunk of data, and outputs another chunk; specifically, it takes in a whole load of business data such as wages, material costs and so on from a configuration file or similar, and produces a single XML file listing products, their prices, product-specific data, data on groups of products, and quite a few other things.

So what should this top-level integration test look like?

I subscribe to the idea of writing top level tests before anything else, acting as a specification of the interface for the code to be programmed - and to a far lesser extent to catch bugs (since at a given level, the set of tests should be more thorough than the level above, so most testing should be done in unit tests.)

I am contemplating writing a single top-level integration test here: feed it a complete configuration file, then test that the XML it outputs is as expected.

So, using PHPUnit, I would have a single TestCase class with a single test() method. And that method would assert every expected XML tag and their contents, and also fail if there are unexpected tags. But only one combination of configuration inputs would be tested. (Or perhaps a couple if it really seems necessary at a later time).

This way, the test would serve as a signpost to keep programming efforts focused on the end goal.

And the test would be easy to read (and maintainable?) since it just consists of a very small test method, an input configuration file and an expected output XML file.

The problem is I have read tests should always be kept small - e.g. at most a handful of assert statements per test method. But then testing a whole XML file like this would be a code smell?

  • 2
    No time for a proper answer, but I think you're on the right track. Only, I would make sure to have at least several of these top level tests. At least one happy path with valid input, and several failing cases with invalid input to make sure you know when you're really done.
    – RubberDuck
    Commented Feb 23, 2016 at 11:42

1 Answer 1


Make sure you make a distinction between integration tests and unit tests. What you're describing is an integration test, where you test the whole thing works when put together. So you'd pass in the XML input and then review the XML output to see if it generated the correct output.

This is different to unit testing. Integration tests can, and often do, take a while to run. Its in their nature to be a lot more encompassing and thorough.

Unit tests should be small and light and fast, but their purpose is to test a small unit of code (eg a class or a function). This way you can change a class and re-run its unit test to ensure you've not broken it.

You can dispense with unit tests if you have a good enough integration test however - most people do not have a good enough integration test to get good coverage of the codebase. Also as integration tests tend to require a lot of deployment or setup and take a while to run, you won't run them very often which can let bugs pass in that will be more difficult to fix if you spot them the moment you make them.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.