66

If I already have integration test for my program, and they all passed, then I have a good feel that it will work. Then what are the reasons to write/add unit tests? Since I already have to write integration tests anyway, I will like to only write unit test for parts that not covered by integration tests.

What I know the benefit of unit test over integration test are

  • Small and hence fast to run (but adding new unit to test something is already tested by integration test means my total test suit get larger and longer to run)
  • Locate bug easier because it only test one thing (but I can start write unit test to verify each individual part when my integration test failed)
  • Find bug that may not be caught in integration test. e.g. masking/offsetting bugs. (but if my integration tests all passes, which means my program will work even some hidden bug exists. So find/fix these bugs are not really high priority unless they start breaking future integration tests or cause performance problem )

And we always want to write less code, but write unit tests need lots more code (mainly setup mock objects). The difference between some of my unit tests and integration tests is that in unit tests, I use mock object, and in integration tests, I use real object. Which have lots of duplication and I don't like duplicated code, even in tests because this add overhead to change code behavior (refactor tool cannot do all work all the time).

2
  • 1
    Just out of curiosity, how much coverage do your integration tests have? Also, if you have some complex logic, do your integration tests cover them all or just a part of it and which parts, important to business or random?
    – civan
    Apr 12, 2017 at 7:20
  • This question seems ambiguous. Let's say I have a Rails controller method that calls an interactor ( github.com/collectiveidea/interactor ). I've decided to write integration tests for my controller method (because without them I can't trust that my API endpoint works). There are at least two questions here: (1) should I also write unit tests for my controller methods (i.e., should I write unit tests that test the same behaviour as my integration tests), and (2) should I also write unit tests for my interactor? I'd answer these two questions differently in certain business contexts.
    – Daniel
    Apr 24, 2019 at 22:22

5 Answers 5

45

You've laid out good arguments for and against unit testing. So you have to ask yourself, "Do I see value in the positive arguments that outweigh the costs in the negative ones?" I certainly do:

  • Small-and-fast is a nice aspect of unit testing, although by no means the most important.
  • Locating-bug[s]-easier is extremely valuable. Many studies of professional software development have shown that the cost of a bug rises steeply as it ages and moves down the software-delivery pipeline.
  • Finding-masked-bugs is valuable. When you know that a particular component has all of its behaviors verified, you can use it in ways that it was not previously used, with confidence. If the only verification is via integration testing, you only know that its current uses behave correctly.
  • Mocking is costly in real-world cases, and maintaining mocks is doubly so. In fact, when mocking "interesting" objects or interfaces, you might even need tests that verify that your mock objects correctly model your real objects!

In my book, the pros outweigh the cons.

6
  • 2
    Isn't mocking cheaper in real-world cases? You're only setting up expectations of what messages the mock receives and specifying what it will return.
    – Dogweather
    Apr 4, 2014 at 22:15
  • 12
    @Dogweather Mocks work well when first created. As time passes and the real object is changed, the mocks have to change with it. 10 years and 6,000 unit tests down the road, it's very difficult to know that your "successful" tests are really successful. Apr 5, 2014 at 12:19
  • 1
    As an aside, that "10 years and 6,000 unit tests" are numbers from personal experience, not hyperbole. Feb 6, 2015 at 3:19
  • 4
    I started writing automated developer tests (unit and integration, but mostly the latter) back in 2004 or 2005, and developed an advanced mocking library for Java. Many times I saw multiple integration tests breaking for the same reason (like a change in the DB, or a network change), and I sometimes write "lower-level" integration tests for reusable components, but still, I much prefer to only have integration tests. Isolated unit tests with mocking, most of the time, are just not worth it; I apply mocking only as an aid for the integration tests.
    – Rogério
    Jul 6, 2015 at 21:57
  • 1
    Your last bullet point seems to contradict the argument you're making. In this bullet point you're making the case against writing unnecessary mocks, which is an argument against writing unit tests for parts that are already covered by integration tests. The thrust of your argument, however, is in favour of writing unit tests for parts that are already covered by integration tests. Could you please clarify your intent here @RossPatterson ?
    – Daniel
    Apr 24, 2019 at 22:13
12

I don't see much value in reeimplementing an existing integration-testcase as a unittest.

Integration tests are often much easier to write for non tdd legacy applications because usually the functionalities-to-be-tested are tightly coupled so testing units in isolation (=unittesting) can be difficuilt/expensive/impossible.

> Then what are the reasons to write/add unit tests?

In my opinion test-driven development is most effective if you write the unit tests before the actual code. This way the code that fulfills the tests becomes clearly separated with a minimum of external references that is easy testable.

If the code already exists without the unit tests it is usually a lot of extra work to write the unit tests afterwards because the code was not written for easy testing.

If you do TDD the code automatically is easy testable.

3
  • 2
    If I have a legacy application with no unit-tests, and I want to include unit-tests for certain parts, do you think its better to first write out the integration tests for the legacy code. Once that is written, then you can uncouple the tight coupling, and create functions with interfaces that can be tested? Since you have the integration-test already written, you can then verify at each step of refactoring, that the new version of the code still works as desired?
    – alpha_989
    Feb 24, 2018 at 23:57
  • 2
    @alpha_989, this is very much IMHO and I'm open to hearing other ideas, but I would prefer Integration Tests over Unit Tests for Legacy Code. In both cases, you would have to be certain that the methods are working correctly before adding any kind of testing, but Integration Testing ensures that the overall expectations of the methods are working as opposed to individual code elements. Mar 29, 2018 at 19:33
  • @alpha_989 yes, that is an approach I have used several times on apps which were lacking test coverage. First I created integration tests which prevented regressions in the important functionalities of the app. Then it was possible to safely refactor the code to be more modular and unit testable without fear of breaking the app. Mar 7 at 16:34
6

Integration tests should only verify that several components are working together as expected. Whether or not the logic of the individual components is accurate should be verified by unit tests.
Most methods have several possible execution paths; think of if-then-else's, input variables with unexpected or just plain wrong values, etc. Usually developers tend to think only about the happy path: the normal path that doesn't go wrong. But as far as those other paths are concerned, you have two options: you can let your end user explore those paths through the actions they take in the UI and hope they don't crash your application, or you can write unit tests that assert the behavior of those other paths and take action where necessary.

5
  • This is true, but a more important question is: do I really need to know that the individual components' complete functionality is working correctly? Or could it be that I've added much more functionality to the component than I really needed for my application? Focusing on low-level components and unit testing can result in a lot of time wasted on dead code. May 14, 2021 at 13:51
  • @FrJeremyKrieg if you follow a TDD approach, you should never have more functionality than you need as you should only write the code that is necessary to pass your tests. If you don't have a need for some functionality, you shouldn't test for it or write the implementation. I agree we shouldn't waste time on dead code - dead code should be deleted immediately! Mar 7 at 16:36
  • @ChristosDimitroulas, your comment is only true for the unit in isolation. Here is a real- world example from personal experience: I had an app for which I needed a parser. I wrote a pull parser, complete with unit tests, using TDD. Then I built my component using the parser, with integration tests, again with TDD. Halfway through I realised it would have been better to use a push parser. So I discarded the pull parser (with its tests) and wrote a push parser without bothering with unit tests - the integration tests remained almost unchanged and proved the push parser's correctness. Mar 9 at 3:36
  • @FrJeremyKrieg there's a lot of debate about what is a unit vs integration test which I don't want to get into. For a parser I would call all the tests "unit tests" personally, even if it's not 100% correct as you probably aren't working with things like a DB. The tests which you didn't have to change when changing the parsers implementation are the perfect level of tests to write - as they don't test implementation details. These are the most valuable type of tests, whether we call them unit or integration is irrelevant :D Mar 9 at 13:03
  • 1
    @ChristosDimitroulas, I agree that the delineation between "unit" and "integration" is somewhat fuzzy. My point was to bear this in mind and not get hung up on writing tests at each "level" - write them at the level where it actually matters. The higher the level requirements that your tests are written at, the less likely you'll end up writing tests you didn't need. :) Mar 10 at 14:11
4

Some of the reasons you've pointed in your question are really important and by themselves could very well make the case in favor of unit testing, but YMMV. For instance, how often do you run your integrated test suite? My experience with integrated tests is that sooner or later they'll get so slow you won't run them every time you make a change and the time between inserting and detecting a bug will increase.

Also, a big mistake you may be making is trusting that

Find bug that may not be caught in integration test. e.g. masking/offsetting bugs.

is not important. Are you expecting that your users find the bugs for you? Trusting coverages obtained from integrated tests is dangerous in my opinion, you can very easily get a high % of coverage but in reality you are testing very little.

I guess the canonical reference against integrated tests are the posts of JBrains:

http://www.jbrains.ca/permalink/integrated-tests-are-a-scam-part-1

in case you haven't read them already.

Finally, IMO the feedback you can get from the unit tests for your design is invaluable. Judging a design by gut feeling and relying on integrated tests may also be a mistake.

2
  • 1
    I don't really get that article. So there are a lot of code paths and integration tests can't cover them all... so? The same thing applies to unit tests unless your units are so trivial as to be pointless.
    – Casey
    Mar 18, 2015 at 19:30
  • That's more like an argument against fixating on code coverage at all than one that you should be writing a ton of unit tests and eschewing integration tests.
    – Casey
    Mar 18, 2015 at 19:31
1

If you expect ever to have to modify or refactor your code, having unit tests is essential. Unit tests exist not only to demonstrate bugs at the time the code is written, but to show when new bugs appear when more code is written.

Yes, it is much better to write your integration and unit tests first, but it there is a lot of value to having them even if they are written later.

2
  • 7
    I don't feel you've made the case. Integration tests would also reveal bugs in the code, and in fact are capable of catching bugs that unit tests will not.
    – Casey
    Mar 18, 2015 at 19:28
  • 1
    Moreover, in addition to what @Casey wrote, when you refactor your units as a result of problems you've run into at the integration level (which happens often), you'll discover that much of the unit code you wrote (along with its corresponding unit tests) was incorrect or even completely redundant/superfluous. This happens less often when you focus your testing efforts at a higher level (eg, integration test). May 14, 2021 at 13:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.