3

Let's say I build my code so that I inject ALL dependencies to every class and when I test, I replace those dependencies with test doubles. Let's also say that when I work on a class I run its tests to test it. Given that a regression is an unexpected failure in a part of the codebase other than that which I changed, it must be due to code (say in class X) which has a dependency on some code I changed being broken by a change in the behaviour of my code. However, class X's dependencies have been replaced with test doubles when it is being tested and those have not changed. Therefore the unit tests on X will not fail. Therefore my unit tests will not catch any regressions caused by my changes.

Is this correct?

(I'm not interested in an explanation of why it is necessary for unit tests to not test dependencies, I'm assuming that's the case and questioning the consequences)

Improve this question
5
  • 3
    Who told you that a regression can only occur in unchanged modules? It's not about place, but about time: the introduction of a defect that was already corrected previously. Therefore regression testing is largely orthogonal to DI.
    – Kilian Foth
    Commented Jun 27, 2018 at 10:25
  • Possible duplicate of (Why) is it important that a unit test not test dependencies?
    – gnat
    Commented Jun 27, 2018 at 10:28
  • 1
    @gnat, that question is seven years old and is full of out-of-date, poor advice answers. And it doesn't even answer this question. So no, it's not a duplicate.
    – David Arno
    Commented Jun 27, 2018 at 10:35
  • @KilianFoth - it's absolutely about place. A regression will have happened at some point in time, obviously - everything does if you want to be literal about it, but the issue is most definitely identifying where the problem is. Which is what OP's asking about.
    – JᴀʏMᴇᴇ
    Commented Jun 27, 2018 at 10:41
  • 1
    @DavidArno: I don't think we can convince gnat to stop him from using his misguided dupe-detection scheme, I have tried this several times, but with no luck.
    – Doc Brown
    Commented Jun 27, 2018 at 11:04

2 Answers 2

Reset to default
6

That is correct. And it is the fundamental flaw with the reductio ad absurdum view that unit tests must only test a single "unit" of code and that everything else should be mocked.

If you adopt the far more sensible approach of view a "unit" as a piece of functionality that can be run in isolation (ie, has no side effects, so can be tested in parallel), then this whole problem goes away. You only replace dependencies with test doubles when needed. If that dependency has a side-effect, or is slow, then push testing that interaction out into your integration tests. Otherwise leave it in place so that your unit tests form useful regression tests.

If you do follow that extreme position on mocking everything, then you'll end having to write lots of integration tests to provide that regression testing. Which then rather begs the question, what are those unit tests for, as they aren't then to provide regression testing?

Improve this answer
4
  • 2
    I don't see a "flaw" in that unit tests must only test a single unit of code. I see only a flaw in believing only unit tests are sufficient for testing a system.
    – Doc Brown
    Commented Jun 27, 2018 at 10:59
  • @DocBrown, "I see only a flaw in believing only unit tests are sufficient for testing a system". Sure I agree with that. Integration tests are needed too. But if your unit test isn't of use for regression testing, then it is of no use as a test. So my "sloppy" way of defining unit tests is a victory for pragmatic logic over illogical dogma.
    – David Arno
    Commented Jun 27, 2018 at 11:07
  • Your statement "But if your unit test isn't of use for regression testing" shows you have a different definition of the term "regression" than the OP. To be honest, I find their definition questionable.
    – Doc Brown
    Commented Jun 27, 2018 at 11:15
  • @DocBrown, I read "regression" as "regression test failure" in the OP. I was going to edit the question to correct this, but wasn't sure if I should. But I answered the question based on that theoretical edit.
    – David Arno
    Commented Jun 27, 2018 at 11:19
2

If you define the term "regression" as a failure in a unit other than the one which was changed, and use the term "unit tests" in the very strict sense for testing isolated units exclusively (and not in the pragmatic way several people often do when they actually talk about automated tests), then you are correct - such kind of unit tests will not find such kind of regressions by definition.

That is why you also also need regression tests, ideally automated ones. And when sticking to your wording, these are integration tests exclusively.

Note, however, most people don't use the term "regressions" in such a narrow sense. A change in one part of a unit can cause an unexpected, unintended change of behaviour in that unit, and the term "regressions" is typically used also for that.

Improve this answer
4
  • 1
    In that case, what would you say was the purpose of automated unit tests? Would you say the practise of running all unit tests after every build was pointless?
    – James Ellis-Jones
    Commented Jun 27, 2018 at 11:28
  • @JamesEllis-Jones: the purpose is to find failures / regressions which don't fit to your very narrow definition of regressions. There is no issue with unit tests, there is an issue with your definition.
    – Doc Brown
    Commented Jun 27, 2018 at 11:37
  • OK so then logically you are saying unit tests are to find failures in the unit that was changed? If the codebase I am describing also follows an interpretation of the SRP that classes should have very limited responsibilities, it is highly likely that change of the class requires change of the tests (except for really trivial internal refactoring). In which case there will be very little valuable reuse of automated unit tests.
    – James Ellis-Jones
    Commented Jun 27, 2018 at 12:50
  • @JamesEllis-Jones: first, in real software units are never so small unit tests won't make sense. And if there is a sensible unit test for a unit, and the unit gets split-up to five smaller units, that won't invalidate the test. The test then has become an integration test instead of a unit test, but it is not better or worse than before. One could say in a codebase constructed of many small units, integration tests may be more important than in a code base with less larger units, but that will actually give a wrong picture, because those are the same tests, just with another name,
    – Doc Brown
    Commented Jun 27, 2018 at 14:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.