Should Integration test offer more value than unit tests? [duplicate]

Question

In order to ask this question I need to define a couple of terms to ensure everyone reading it is on the same page:

• Unit testing is the testing of a single class with all dependencies mocked / stubbed in some sort of way.
• Integration testing, in this context, means integration of a selection of objects within a system with some potential mocked dependencies. For example external collaborators (such as a database or remote third party) may be mock but then internal systems may also be mocked (e.g. mock out the authentication mechanism to always work). The idea is to spin up some non trivially sized part of the system and verify it behaves.
• Functional testing is spinning up the system as a whole and poking it to achieve the desired results. For the purposes of this question it may be against real external coll

From the test pyramid perspective we should have 70% unit to 20% integration to 10% functional tests.

With the above out the way my question is as follows : I've worked in environments where there are a large suite of unit tests however bugs still surface at runtime due to insufficient integration testing / non existent functional testing. Even with excellent unit coverage of a given class how that class is actually used / how it is wired into the rest of the system typically contains more bugs than you can really catch at the unit level. Usually these bugs are detected at the integration / functional testing level.

Granted with unit tests you can put the system into more states and they tend to run a lot quicker but as a bug detector they fall down compared to integration / functional level testing that actually drives code paths. When I raise this point with others the usual response is to say "oh, the integration tests are actually finding unit tests you didn't write!", this seems valid at first glance but all it means is I'm now testing my code twice : once at the integration level with full coverage and once at the unit level where I can't be guaranteed of the level of coverage. However I need to maintain both sets.

My question is if integration test setup is kept to a minimum / abstracted away in a common test setup library then why not use integration tests as the 'first line' of testing over unit tests? What's the actual value of testing everything twice?

Answer 1

This is thing A. Thing A gives 2 when you ask it the count after clicking it twice. You know thing A works this way because you just saw code that made it do that and watched it do that. You feel like you understand what A is supposed to be without even having to look inside.

Here's system abcxyz. When started it generates a gif that looks like an odometer (or a score board, or a take-a-number display). It can be configured many wild and wonderful ways. But when looking at the integration test code that exercises it... well... it takes you a while to figure out what it expects, what makes it count up, where it's getting these images. You think you know what this is supposed to be but the moment you go to make changes you realize you need to understand what's using abcxyz a little more. So you look at the rest of the system. You keep reading... and reading... sigh...

I like having both tests. I like the confidence that a unit test gives me that I understand what the author really thought was important to A. What the author thought the outside world would expect. I actually find well done unit tests easier to read then the class definition.

I like to turn around and use that to inform my understanding of integration tests. Doing so without unit tests feels like eating your 4 course dinner after it's been put in a blender. You can do it. But yuck.

And that's just what it's like to read the code, let alone touch it.

Never trust things that make writing code easy to tell you anything about how to write code that is easy to read.

Answer 2

It is a question of top-down versus bottom-up development. When you are building a big machine, do you build the whole system first and then test it as a whole, or do you develop and test each component individually before assembling the whole? The correct answer is you do both. Certainly you have to test the system as a whole, but there are also numerous benefits to testing components individually:

• You can develop components individually and in parallel.
• You discover errors earlier (you don't have to wait until all the whole machine is finished before testing).
• It is much easier to pinpoint the error when testing components individually.
• You can test edge cases and error conditions which might be impossible to replicate in the system as a whole (because of other safeguards).
• The tests are much simpler to write.
• Testing components individually forces you to design each component with a well-defined interface and behavior and minimal number of dependencies.

Answer 3

The benefit of the increasing detail level of tests is that they can identify the exact problem, rather than just the existence of a problem.

Also, you should think about testing requirements and about defining requirements for individual parts of the system that you might unit test.

For example, say I have a BasketPriceCalculator which returns the price of baskets of products based on offers and things. If I put in a unit test for the 2 for 1 offer and it fails, then I know there is a problem with that offer in that class. Where as the equivalent integration or functional test might fail if any component in its chain failed.

Also, I might have a specified requirement that the BasketPriceCalculator throw an error if there are more than 7 items in the basket. I would want to test that requirement without including dependencies, because its specific to that component, not the product as a whole, which might handle the error an take some other action.

On the whole I agree with you. Integration tests are more important than Unit tests and there isn't really a reason why you shouldn't run every unit test with mock mocked and real dependencies. Which I do with data driven tests.

Functional tests are even more important and again, even though they are slow to run there isn't really a reason not to run every possible scenario you can think of as a Functional test. After all your users will.

But practically speaking, when you are developing something from scratch, you don't start with a functioning app that you can write a functional test for. you start with some small component which calculates the price of a basket based on some rules you have been given. So you write unit tests for that bit first.