2

I've been trying to understand when to mock and when not to mock, however I'm not able to come up with a consistent guideline and I'm hoping to get some input on the subject. Let's look at the following toy example:

class ServiceMixer:
  def __init__(...):
    self._expensive_rpc_service = ExpensiveRPCService(...)
    self._db_scan_service = DBScanService(...)
    self._cpu_intensive_computation_service = ExpensiveCPUService(...)

  def get_results(input):
    result_1 = self._expensive_rpc_service(input)
    result_2 = self._db_scan_service(input)
    result_3 = self._cpu_intensive_computation_service(input)

    return compute_output(result_1, result_2, result_3) 

I want to now write unit-tests on the get_results method. I do understand that in this example, I will need to use a mock somewhere regardless - two of the services have side-effects (i.e. they talk to a DB or make an RPC to another service). The question is, should I mock the 3 services shown here, or should I mock the RPC calls and the DB calls that the services make in the ServiceMixer unit tests?

One advantage of mocking just the RPC / DB Calls is that I can test the contracts between the difference services and the ServiceMixer. If a downstream service changes, then the service mixer test will break. This is somewhat like an integration test. On the other hand, it will mean that the unit tests for get_results need to innately understand the nuances of how each service works so it can correctly place expectations of the arguments + returned values.

If I were to just mock the services here, then I can just focus on testing the behavior of the ServiceMixer, but I will then miss out the caller-callee contract testing. So, what is the best practice here? It seems like using mocks is the right approach, but I've seen a few articles that state "too many mocks are a code-smell". When is it appropriate to use a mock? When is it not?

Thanks!

2
  • 2
    In which scope does compute_output live? Can you unit-test it without any mocking?
    – Doc Brown
    Jan 25 at 8:59
  • Here's a PyCon talk that discusses the tradeoffs between using mocks and other strategies when testing: Stop Using Mocks (for a while).
    – jfaccioni
    Jan 26 at 12:12

3 Answers 3

3

When you unit-test class X, you should mock all the collaborators of X unless they're trivial and don't add significant cost or complexity to the test execution.

So when testing ServiceMixer, you should mock all three collaborator services. The actual computation/DB lookups that the entire algorithm is about should play no part in your test at all; it should only test that ServiceMixer orchestrates things correctly.

When you unit-test, say, RPCService, you would normally mock the external part of the computation and test only that your service issues the correct RPC commands to the external system. You should not verify that the external system does its job correctly; that is important, of course, but it's the job of an integration test.

Keeping unit-level testing separate from system testing allows you to distinguish problems with a specific component from problems with the interaction between parts. This makes software engineering faster and more accurate. "Divide and conquer" is a thing, and it is invaluable when your system grows beyond a certain size - usually a surprisingly small size.

0

You are thinking about it too much. The value added by any test is clearly visible when they fail. You should know what went wrong and where you need to work for a fix just from seeing which tests fail. The easy way to get this right:

  • Do you want to check if your code works correctly? Then mock everything that's providing data to your code.
  • Do you want to see if you code works together with other components? Then you should mock everything else.
  • Do you want to see if the whole app / systems works correctly? Then don't mock anything...

When it is hard to make this separation it means you should redesign your code to make that decision and implementation easier.

Just start small and keep them growing together: a little bit of production code and a little bit of tests. Does this ring any bells?

2
  • 3
    It can also be hard to make the separation if you try to apply both test goals you mentioned in a single test. Then you should split your test, rather than redesign the code. Jan 25 at 8:57
  • Fair enough, that worths to be mentioned, each test should have one goal (the SRP applies to test code too).
    – Florin C.
    Jan 25 at 9:00
0

I've been trying to understand when to mock and when not to mock, however I'm not able to come up with a consistent guideline

That's not your fault - the literature is a bit of a mess here. This is both because there are a lot of similar patterns that will sometimes wear the label "mock", and because people use mocks to solve different kinds of problems.


I want to now write unit-tests on the get_results method.

Serious question: why not test the compute_output method instead? After all, that's where the complexity is located?

get_results, as shown here, is "so simple there are obviously no deficiencies". The only significant risk I can see is that you might pass the results to compute_output in the wrong order, and it's probably more cost effective to use visual inspection/code review to detect that problem.

(Possible answer: you are worried about the order of arguments in compute_output changing, and you are hoping that this test catches that problem. I'd suggest that's an inferior approach to adding tests to compute_output to ensure that future changes don't break backwards compatibility).


should I mock the 3 services shown here, or should I mock the RPC calls and the DB calls that the services make in the ServiceMixer unit tests?

There are different answers, depending on what you are trying to measure.

As a general rule, the more "real" modules included within your system under test, the more likely it is that the test will span a module that has a change to its requirements, which makes it more likely that the observable behavior of the system will change, which in turn makes it more likely that the test will need to be rewritten.

See Parnas, 1971.

On the other hand, as you add more "real" modules to the system, the behaviors observed in test become a more reliable indicator of what's going to happen in production.

For a "programmer test", the primary use is to measure whether the most recent refactoring broke this piece right here. That means we want the behaviors of the dependencies to be stable, which in the general case will argue for introducing stable substitutes for the direct dependencies where necessary.

For a "customer test", you'll normally want to pull in as much "real" code as you can, subject to other constraints. The idea here being that you know you've got a collection of modules that each pass their own "programmer tests", and you are verifying that they deliver value when they are all wired together.

"Horses for courses."

Kent Beck's Test Desiderata essays (2019-06, 2019-10) may be helpful in understanding the tradeoffs you need to consider.


When is it appropriate to use a mock?

There are two common cases:

One: any time where using the real implementation makes the testing costs too "expensive" (which can include simple things like test latency). If you are expecting developers to run the tests while refactoring, then you want execution times on the order of 300ms (see Bernhardt 2014). Introducing substitutes that eliminate the need for I/O, network, filesystem, and so on is a way to reduce execution time.

Two: when you are designing a system where objects send each other messages, and you want to verify that the correct message is being produced without worrying about how that message is consumed. See Mock Roles, Not Objects, or Growing Object Oriented Software, Guided By Tests (Freeman, Pryce). In other words, mocks are introduced as a mechanism for designing the protocols used to pass information from one object to another.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.