4

I feel like, in web dev at least, that integration tests can't really find any useful bugs. I can't think of any, at least. If I can use unit tests to verify that one section of my code calls a certain method, and that another section of my code responds properly when it receives the exact same input, then what's the point of integration testing? I'm looking mainly for some types of examples of bugs it can find. Something more than "issues with the connection between such-and-such part of the code and such-and-such part of the code".

  • 1
    Sometimes your mocked objects don't properly simulate what they're replacing. Integration tests help detect that. – whatsisname Mar 10 '16 at 6:35
  • 5
    It's the bugs that you don't expect that make integration tests so useful. – Robert Harvey Mar 10 '16 at 6:52
11

I'll give you 2 examples:

A trivial one, I once saw a network class that allowed you to set the host and port and then send messages. Unit testing worked - you could set the host, you could set the port, you could call the method that sent messages.

On integration, somebody wrote the client code that used this class and set the port before setting the host. Nothing worked, because the set host method reset the port to a default.

That's a simple, simple example that shows your unit tests can show correct behaviour, but because the methods were not called in the expected way, it failed at runtime. You could argue the unit tests were not correct, in that they didn't test every combination of the property setting calls, but unit test is to ensure each section of code works correctly (eg sets the right property) and not how the interaction between then occurs. (Of course, you would be better to argue that the class should have been the unit of test here and that might have caught this bug, but even so I doubt anyone would have repeated the test with different combination of property setup).

Another example is with threading. I have had a class that worked perfectly in unit testing, it takes a lock on a shared resource and calls a method on a different object, which as that method is mocked, returns immediately. In the real world, that method actually does a lot more work, creates a new thread that waits for the same shared resource to become available, and thus deadlocked the whole program. Good luck finding that in unit testing.

Unit tests are not the primary testing you should do. In fact, if you had to only have 1 layer of testing it would be integration testing. Unit tests are a "quick and dirty" way of getting past the obvious bugs raised during development and are more of a developer check to make sure you didn't do anything stupid (like the time I wrote an arithmetic class that added when you called the subtract operator.. a unit test would have caught that cut and paste laziness). That's all they should be there for though, trying to ensure perfect system operation using them is misguided.

8

Integration tests are the first tests in which two or more real (i.e. not mocked) system components interact with each other. If you have both unit- and integration tests, then a failing integration test is a strong indicator that the behaviour of the mock for Component A does not correspond to the behaviour of the real Component A.

Especially in a larger system, it is easy to create a disconnect between the behaviour of the real component and the expected behaviour that is encoded in the mocks and their usage.

4

It does depend a bit on the system you're testing, but they can find lots of things that aren't covered by nice neat stateless unit tests.

For me the really big one is talking to the database. You can have your unit tests on things that decide to talk to the database, and you can have your unit tests on things which run inside the database (sometimes, with some dirty tricks), but you need integration tests to make sure these things can all talk to each other properly. The database/application interface is generally quite fragile, so it's important to have tests over as much of it as you can possibly manage.

Systems containing multiple components communicating over REST or a message bus benefit enormously from integration tests to verify that components can actually work with each other in the way they're supposed to. They might both have comprehensive unit test coverage on their external interfaces, but does the Frobnicator's idea of the Mungeifier's public interface match the public interface the Mungeifier actually provided? Even the most loosely coupled components imaginable have to conform to some kind of interface contract or communication becomes entirely impossible. This all has to be tested, and the only way to be sure is to test it for real.

Also, you can view performance testing as a kind of integration test. Not only are you testing if the system can perform under expected loads, because you'll be testing the entire system as assembled for production (or you should be), you're also doing integration testing on the performance workload. Thus your performance test harness needs to be able to spot behavioural errors as well as performance characteristics. It's no good being fast if you're wrong!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.