0

Given: Let's say I have 3 objects A, B, C that form a cluster. Each unit (A,B,C) is independently tested with mocked collaborators. The "integration" is the call to A that calls out to B and C based on some conditions (e.g. either both are called or only B is called, say).

Testing Approach:

  • Unit test for A relies on mocking interfaces of B, C and testing that A interacts with the interface correctly.
  • Unit tests for B, C, relies on the correct data being passed into their method(s) and the tests actually are testing B, C against their interface "contract". Assumptions made by A are being tested for in tests for B, C

For the sake of simplicity, let's assume, neither unit makes any calls to databases or other systems...everything is "in memory".

Problem: A senior engineer on the team says: You haven't really tested that the overall functionality exposed by A is indeed working! Everything is testing against mocks!

Argument: One can write a simple "unit integration" (pardon the term) for a simple scenario as a sanity check if needed but it isn't really necessary if we're testing like the above, no?

Question: What is the flaw in the argument above and how to best answer the engineer's question?

From my POV, the tests are "technically complete" making it rather unlikely that once concrete implementations of B, C are wired into A, there's a chance something will break. What's the best way to frame the counter argument that makes such a "unit integration" test unnecessary (or what's the flaw in my understanding that I need to update)?

The above is a simplified view of the way we have testing structured in the repo where the "outside world" is only interacting at the "edges" and the core of the business logic is entirely unit-testable.

7
  • If there is a request for close, I'd really appreciate a reason as it'd provide me an opportunity to reflect and possibly clarify and possibly delete/close the question myself. I'm aware it's not necessary to provide a reason, but I'm humbly requesting for one.
    – PhD
    Jan 18 at 18:41
  • I see a vote to close this as opinion-based. It wasn't me, but I don't know if we can identify a single recommendation here. There is definitely an element of subjectivity here, but I'm not sure if it really means this question is opinionated. Jan 18 at 18:50
  • I personally do not think this question is opinion-based. The key here is: 'The "integration" is the call to A that calls out to B and C based on some conditions'. Jan 18 at 18:52
  • Could you edit your question to include some example or pseudo code to illustrate this "integration"? Jan 18 at 18:54
  • The tests are meant to answer question "When tests are green, do you feel safe about deploying into production?" If adding the integration test makes you feel safer, then do it. If not, then do not.
    – Euphoric
    Jan 18 at 19:02

1 Answer 1

2

The justification I can see for writing an integration test using concrete instances of A, B, and C is to guard against assumptions made in tests drifting out of sync with the concrete B and C classes. To be honest, I was expecting that B or C were making calls out to a database, web service, or the file system.

If interfaces for all three classes exist because you are leveraging polymorphism, then there is nothing wrong with wanting example tests using the concrete objects to verify the cluster is functioning together. Tests for each class individually will make assertions based on assumptions about how those collaborating objects interact. If you cannot verify a change in one of these collaborators using a compiler, then there is a chance the logic of B and C could drift away from the expectations defined by tests for class A. This is simply the price you must pay for utilizing polymorphism.

If interfaces were created so you can mock things in testing, and there is no communication to the outside world, why even have interfaces? Use the real things throughout. That would be the design flaw causing your colleague to ask for additional tests.

So, my advice is twofold:

  • If you are actually using polymorphism, your coworker is right. You'll need to add integration tests for all concrete actors. This is a reasonable request, in my opinion. You don't need many tests, though. Just enough to feel confident that the cluster of classes is cohesive.

  • If these interfaces only exist so you can mock things in unit tests, get rid of the interfaces. Everything executes in memory inside a single thread. The interfaces are unnecessary. Your tests will verify the real behavior, which makes your coworker's request redundant.

5
  • 1
    Yes. I understand the need for sanity checking the polymorphic aspect. That was my thinking too and I could see where he's coming from. Hence, I added simple tests that helped verify "everything seems well connected and behaves as expected" without duplicating what existing tests are already doing.
    – PhD
    Jan 18 at 20:52
  • The 2nd part: using concretes directly. There are parts in the code base that have this. The tests for such "integrations" (like A) makes each test needlessly "big" - the test for A almost subsumes the tests for B and C and most folks are unable to separate the unit from the integration IMHO. Yes, we can write blackbox tests that can simply "check" the behavior of A and conclude B and C were correctly called under the hood. That is what I'm helping the team understand vs. defaulting to only writing integration tests (unit-integration + real).
    – PhD
    Jan 18 at 20:55
  • @PhD: when tests for A subsume tests for B and C, it indicates that B and C are implementation details of A. They might not need their own tests. Not everything needs its own test suite separate from everything else. Jan 18 at 22:19
  • That may be true. However, these "details" could have numerous code paths and testing them as part of A can lead to much larger tests than strictly necessary. If B has 3 code paths, C has 2 and A has 4 (say), when testing A (as B, C are details and hidden), one would need 4 * 3 * 2 = 24 tests! However, this can be brought down to 4 + 3 + 2 = 9 if one breaks it out. Even if we add a sanity check "unit integration" test like I mentioned, it's still MUCH smaller IMHO. Unless I'm missing something, such "details" might as well be their own abstractions that haven't been taken out.
    – PhD
    Jan 19 at 0:15
  • @PhD: You don't need to test every possible combination of paths. If you already tested the paths individually (i.e. 4 + 3 + 2 = 9 unit tests), then you can get by using a reasonable amount of scenarios in your integration test. Depending on the context (and the cost of making mistakes e.g. for medical equipment), that may justify doing the lot, but in less severe cases you're generally going to only test a few common sense scenarios to confirm that the integration (not all individual paths themselves!) work as expected.
    – Flater
    Jan 19 at 0:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.