0

How many dependencies does a unit test have (that are not mocked)?

My understanding is that a true "unit test" has zero dependencies, with all of them mocked (or none in the first place; pure functions.) (Unit tests are excellent for pure code, but in a real application most code is not that pure. They work great for string manipulation functions, formulas, etc.)

Beyond this are "integration tests", where the dependencies are real and the test makes sure they can work together.

I feel like missing in between these are tests of 1 dependency (whether these are normally called "unit tests" or "integration tests" I'm not sure.)

For examples, I want to test the UI itself without the DB, and the DB itself without the UI. I want to test that my report executes, without putting it in a window. I want to test that my resource loads without having to put it somewhere; I want to test that the window can display a resource, without having to load a real one. Once I know these things, there's very little if any room left for problems to emerge at runtime or in an "integration" test when all dependencies are real.

These seem to be the tests that actually tell me that my application works, and so give me the most value for my effort. Tests of 0 dependency are sometimes useful for truly pure code, but for the vast majority of my code, tests of 1 dependency seem to be the best. And if I mocked out that 1 dependency, then the tests would no longer be telling me anything useful, since they would not really be testing anything.

Does this make sense, and/or am I missing something or do I have something wrong here? Are 1-dependency tests considered unit tests, and if not, why are "unit tests" considered so important when they don't really test your application.

3
  • Not sure whether it quite answers the question, but I think this related question would be worth reading: softwareengineering.stackexchange.com/questions/244705/… Oct 1, 2022 at 20:37
  • You can have a pure function with n dependencies. E.g. a pure function that filters a collection, takes in a lambda that tells it which elements to keep (also a pure function), and returns a new collection containing only the elements that passed the test. The lambda is a dependency. If you wrote tests for the filter function, you'd create some test data, and inject an ad hoc, test-specific lambda predicate. That predicate is technically a mock (or rather, a stub). Note that the test is not testing (and it's not supposed to be testing) the lambda (or, more generally, any mock). Oct 2, 2022 at 7:39
  • @FilipMilovanović I think I am considering "dependency" here at the module-level. My logic library is 0-dependency, and my other libraries are 1-dependency (where dependency is something like UI, DB, resources, CLI, etc.) I want to know the UI works, but I don't want an active DB while I test it. Theoretically I could abstract out the entire UI such that my UI code could be executed with no real UI components, but if that's the idea I haven't made it that far yet. Oct 2, 2022 at 16:59

3 Answers 3

1

"why are "unit tests" considered so important when they don't really test your application"

Perhaps an analogy will help; an isolated unit test is like SpaceX engineers testing one of their Raptor engines without hooking it up to an actual rocket.

Instead, they hook it up to test equipment which allows them to test the behavior of the engine in isolation, and allows them fine control over the experimental conditions.

The engine is the unit under test, and the supporting equipment is, quite literally, a mockup of the other parts of the rocket that connect to the engine. It doesn't do everything that the rocket does. It does, however, have to do everything that's necessary for the purposes of the test, and it does have to interface with the engine in the same way a rocket would (it has to have the same kinds of connectors and/or adapters to hook up to it in the same way).

Note that the engine is not actually doing the job that it's really supposed to do (put a rocket into space, and help land it safely), but they can still test, to a significant extent, if they made it right (if it connects properly, responds to controls, generates the right kinds of outputs, the right amount of thrust, etc).

A unit test should be very much like that. You "poke" the function/class/component ("unit") under test in different ways, and you check if, externally, it appears to adhere to the logic in the way you expect it to. So, in that light, the test should be a stand-in for the actual production code that's going to be interacting with your unit; it's a tightly controlled representation of the immediate "environment" of the unit. If such tests are hard to write, then your actual unit will be difficult to use and/or reuse in production. If the tests fail, then the code using your unit in production will likely fail too. The idea is that both the actual client code and the tests rely on the same things. Mocks help you control the "environmental" conditions and "measure" the way the unit responds to various "pokes". They are test-specific, and you configure them to do exactly what you need them to do for that specific test, and nothing else.

SpaceX engineers - and we, software developers - do it because, if done properly, it's much cheaper and faster to test this way. You don't get all the info, but you get a lot of fundamental info that gives you confidence that parts of your system do what they are supposed to do at the component level.

This means that you can run experiments more frequently, and that you can get feedback much more quickly (= unit tests should be lightweight and super-fast). In our case, the feedback is (1) on the correctness of the behavioral contracts that the component is supposed to adhere to, and (2) on how usable the interface (the API) of the component is.

You get quality by failing fast, and failing as many times as it takes you to figure out the best way to do it. That doesn't mean it's easy coming up with good tests, or with good design for the unit, and the system itself.

Actually attempting to fly the rocket is an integration test (they probably have levels of integration experiments between testing the engine in isolation and going to the launch pad, but let's go with this analogy). What you're trying to do there is look for any unwanted behavior emerging from the interaction of all the parts.

This test is expensive, and in comparison, it's not at all quick and easy to set up. Countdown: 3, 2, 1. Wooooshhhh! We have a lift of! ... A few seconds later, the rocket blows up. This is a good thing. It allows you to catch problems early, learn about them, and try and create a more robust system. In some sense, SpaceX was able to do the amazing things they've done because they could afford blowing up a bunch of rockets. Heck, they blew up a rocket on purpose to test the abort mechanism.

A bad thing would be if a rocket blows up later on - you know, when in production.

"I feel like missing in between these are tests of 1 dependency (whether these are normally called "unit tests" or "integration tests" I'm not sure.)"

There's a little bit of a blurred line there. In one of the preceding paragraphs, I used the phrase: 'You "poke" the function/class/component ("unit")'. A unit is what you define it to be - and it doesn't have to be the same thing everywhere throughout the same project. You're looking for something that you can call a "component" that has a narrowly defined job, with well defined behavior that you can easily and quickly test in isolation (that is, without dependencies external to the "component", as you defined it).

A "component" could be a single function, or an object, or a small bundle of interacting objects. Such a bundle can create a couple of objects internally - your tests (for that bundle) should not know about them (for the same reasons actual client code in production should not know about them). Such a bundle can also have one or more external dependencies that you can inject at construction time - these are part of its client-facing API - and your tests should mock them.

So, thinking in terms of the number of dependencies is not a good way to differentiate unit tests from integration tests.

Here's a set of guidelines coming form Michael Feathers, the author of Working Effectively with Legacy Code (emphasis mine, [comments] mine):

A test is not a unit test if:

  • It talks to the database
  • It communicates across the network
  • It touches the file system
  • It can't run at the same time as any of your other unit tests [meaning, tests aren't independent of each other]
  • You have to do special things to your environment (such as editing config files) to run it.

Tests that do these things aren't bad. Often they are worth writing, and they can be written in a unit test harness. However, it is important to be able to separate them from true unit tests so that we can keep a set of tests that we can run fast whenever we make our changes.

I want to emphasize that all of this is not to put the tests on a pedestal. It's not about tests, it's about the design of the code under test. It's about applying your design skills. Tests are a tool that helps with that. Here's another thing I want to quote Michael Feathers on:

"Solving design problems solves testing problems1. Making the code testable2 doesn't necessarily make the design better."


1 Allows for testability.
2 E.g. making a method public just for the sake of testing.


Tests starts falling into the integration test territory when you stop testing if a component adheres to well-defined behaviors associated with it by design, and start testing to see if "stuff works together". Note that integration tests, generally speaking, can have varying scope (integration of a couple of components, or a larger part of the system, or the entire system). The way you write these tests and gather data might be a bit different compared to what you'd do in unit tests (e.g. you might start using different tools as the tests grow in scope, a QA person might be involved, etc). Also note that people in the industry use differing definitions for what constitutes an integration test - it's not a sharply defined term industry-wide.

According to the test pyramid, you'd have a smaller number of these tests compared to unit tests.

enter image description here

4
  • I would say that the SpaceX example is (maybe) a test of 1-dependency, because you have a real engine with a mock rocket. (Not sure physical example is good because with software we have interfaces that let us pull some or all of the logic out of the "real" implementation. Theoretically we would mock the "heavy" parts (network, file access), and still be able to test the engine logic or rocket logic separately. It would be like knowing we have a working engine without using fuel.) Oct 2, 2022 at 16:46
  • My question is, what are "tests of 1-dependency", unit tests, or integration tests? Why aren't they differentiated from tests of 0-dependency. This seems to me like a crucial distinction, but in the communication of tests, this is never brought up. Always the distinction is unit vs integration, which seem to be 0-dependency and N-dependency respectively. Oct 2, 2022 at 16:46
  • (It's possible that my code-base/code-style does not well support the kinds of testing that you're talking about, so my problems are specific to the way my code is written and you would design these systems differently in ways that I've not yet been able to.) Oct 2, 2022 at 16:51
  • I notice that the comments on that Michael Feathers article are basically arguing about this exact concept; MF says that a 1-dependency test is an integration test (it tests integration of your code with the dependency), whereas others are saying that they're still unit tests and are essential to tell if your systems actually work. MF's only argument against them seems to be the speed of running your tests. Oct 2, 2022 at 17:20
1

How many dependencies does a unit test have (that are not mocked)?

How many unreliable dependencies does a unit test have? 0.

But that constraint does not apply to reliable dependencies. For example, in Java, there's nothing wrong with having a dependency on Java Collections, because when our test reveals a fault, we have lots of nines of confidence that the fault won't be in the collections library that everybody has been using since forever.

If Foo depends on Bar, and we've already verified Bar, we can use Bar in our "unit" test of Foo without disturbing the test properties we care about.


I feel like missing in between these are tests of 1 dependency (whether these are normally called "unit tests" or "integration tests" I'm not sure.)

Beizer 1990 talks about "Component Testing", where the test subject is an integrated aggregate of one or more units.

Integration testing should not be confused with testing integrated objects, which is just higher level component testing.


"Unit test" is not a single common ideal - we have a lot of different contexts in which that phrase appears, and usually several different opinions in each context about what is important.

I call them "unit tests" but they don't match the accepted definition of unit tests very well -- Kent Beck, 2003


These seem to be the tests that actually tell me that my application works, and so give me the most value for my effort.

Which is why you should keep doing that. Does the label we use to classify those tests matter so much?

0

Yes and No.

If you only do "end to end" tests. ie you emulate a user using your program on a fully setup test bed with all dependencies, yes you detect when problems happen at any layer, but! there are downsides.

  1. You know something is wrong but not what is wrong.
  2. Your tests are hard to write, you need to cover different combinations of variables. "will it break", "will is break in dark mode" etc
  3. Your tests are slow. If you have to spin everything up and emulate clicks and wait for things to happen running a whole test suite can take hours.

If you only write "pure" unit tests, then you can write a lot of boring tests that don't really test anything, except your mock. So you need to strike a balance.

  • Write pure unit tests for complex logic that you might have programmed wrong.
  • Write integration tests for simple stuff which relies on lots of layers being setup properly
  • Write end to end tests for your critical paths. "Buy a product", "Login", "View home page" etc
7
  • thanks, but you're still jumping back and forth between 0-dependency tests and N-dependency tests, when my question is specifically about the value of 1-dependency tests, and whether they are considered unit or integration tests. Oct 1, 2022 at 22:20
  • 4
    I think the common practice now is to de-emphasise what you call the test and concentrate on whether its a useful test. 1 dependency isn't a magic number, you could have 2 or 3 very thin layers of code and still want to test them together, Or tightly coupled parts which you don't want to decouple for "pure" tests
    – Ewan
    Oct 1, 2022 at 22:59
  • I guess you're saying that at both extremes (too many, not enough) maybe testing can become less valuable, so maybe you're agreeing that there's a sweet spot in the middle Oct 2, 2022 at 0:55
  • @DaveCousineau I think it depends on your definition of a dependency. Kent Beck's original TDD book tries to emphasise testing behaviour (e.g. relating back to use cases, requirements, API contracts, etc.). Not having tests rely upon code structure, so if part of a behaviour exists in other classes then those classes aren't dependencies but part of the same unit/behaviour and shouldn't be mocked or tested separately. In that sense unit tests should have 0 dependencies since a dependency implies something external to the behaviour under test. Oct 2, 2022 at 7:16
  • 4
    hmm, no, I'm saying its not true that number of dependencies alone determines test usefulness. In some cases 0 dependency tests are the best, in some they are not useful. You cant pick a "best number of dependencies" for all testing
    – Ewan
    Oct 2, 2022 at 8:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.