I titled the question jokingly because I'm sure that "it depends," but I have some specific questions.
Working in software that has many deep layers of dependency, my team has become accustomed using mocking fairly extensively to separate each code module from the dependencies beneath it.
Therefore I was surprised that Roy Osherove suggested in this video that you should be using mocking only something like 5% of the time. I'd guess that we're sitting somewhere between 70-90%. I've seen other similar guidance from time to time as well.
I should define what I consider to be two categories of "integration tests" which are so distinct that they really should be given different names: 1) In-process tests which integrate multiple code modules and 2) Out-of-process tests which talk to databases, file systems, web services, etc. It is type #1 that I am concerned with, tests which integrate multiple code modules all in-process.
Much of the community guidance I have read suggests that you should prefer a large number of isolated, fine-grained unit tests and a small number of coarse-grained end-to-end integration tests, because unit tests give you precise feedback on exactly where regressions might have been created, but the coarse tests, which are cumbersome to set up, actually verify more end-to-end functionality of the system.
Given this, it seems necessary to make rather frequent use of mocking to isolate these separate units of code.
Given an object model as follows:
... Also consider that our application's dependency depth goes much deeper than I could fit in this image, so that there are multiple layers N between the 2-4 layer and the 5-13 layer.
If I want to test some simple logical decision being made in unit #1, and if every dependency is constructor-injected into the code module that depends on it such that, say, 2, 3, and 4 are constructor injected into module 1 in the image, I would much rather inject mocks of 2, 3, and 4 into 1.
Otherwise, I would need to construct concrete instances of 2, 3, and 4. This can be more difficult than just some extra typing. Often 2, 3, and 4 will have constructor requirements which can be challenging to satisfy and according to the graph (and according to the reality of our project), I'll need to construct concrete instances of N through 13 to satisfy the constructors of 2, 3, and 4.
This situation becomes more challenging when I need 2, 3, or 4 to behave in some certain way so that I can test the simple logical decision in #1. I might need to understand and "mentally reason about" the entire object graph/tree all at once to get 2, 3, or 4 to behave in the necessary way. It often seems much easier to do myMockOfModule2.Setup(x=> x.GoLeftOrRight()).Returns(new Right()); to test that module 1 responds as expected when module 2 tells it to go right.
If I were to test concrete instances of 2 ... N ... 13 all together, the test setups would be very large and mostly duplicated. Test failures might not do a very good job of pinpointing the locations of regressions failures. Tests would not be independent (another supporting link).
Granted, it is often reasonable to do state-based, rather than interaction-based, testing of the bottom layer, since those modules seldom have any further dependency. But it seems that mocking is almost necessary by definition to isolate any modules above the bottom-most.
Given all of this, can anyone tell me what I might be missing? Is our team overusing mocks? Or is there perhaps some assumption in typical unit testing guidance that the layers of dependency in most applications will be shallow enough that it is indeed reasonable to test all of the code modules integrated together (making our case "special")? Or perhaps differently, is our team not bounding our bounded contexts adequately?
Or is there perhaps some assumption in typical unit testing guidance that the layers of dependency in most applications will be shallow enough that it is indeed reasonable to test all of the code modules integrated together (making our case "special")?<-- This.