13

Some time ago I read, on a Stack Overflow answer that I can't find, a sentence that explained that you should test public APIs, and the author said that you should test interfaces. The author also explained that if a method implementation changed, you shouldn't need to modify the test case, as doing this would break the contract that make sure the system under test works. In other words, a test should fail if the method doesn't work, but not because the implementation changed.

This called to my attention when we talk about mocking. Since mocking relies heavily on expectation calls from system under test's dependencies, mocks are tightly coupled with the implementation rather than the interface.

While researching mock vs stub, several articles agree that stubs should be used instead of mocks, as they don't rely on expectations from dependencies, meaning the test need no knowledge of the underlying system under test implementation.

My questions would be:

  1. Do mocks violate the open/closed principle?
  2. Is there something missing in the argument in favor of stubs on the last paragraph, that make stubs not so great vs mocks?
  3. If so, when would be a good use case to mock and when would be a good use case to use stubs?
  • 4
    Sometimes, the test justifies the means – user40980 Aug 12 '15 at 22:03
  • 8
    Since mocking relays heavily on expectation calls from system under test's dependencies... I think this is where you're going awry. A mock is some artificial representation of an external system. It doesn't represent the external system in any way, except insofar as it simulates the external system in such a way that it allows tests to be run against code having dependencies on said external system. You'll still need integration tests to prove that your code works with the real, unmocked system. – Robert Harvey Aug 12 '15 at 22:27
  • 8
    To put it another way, the mock is a substitute implementation. Which is why we programmed to an interface in the first place, so that we could use mocks as a stand-in for the real implementation. In other words, mocks are decoupled from, not coupled to, the actual implementation. – Robert Harvey Aug 12 '15 at 22:28
  • 3
    "In other words, a test should fail if the method doesn't work, but not because the implementation changed", this is not always true. There are plenty of circumstances where you should change both your implementation and your tests. – whatsisname Aug 13 '15 at 2:06
4
  1. I do not see why mocks would violate the open/closed principle. If you could explain to us why you think they might, then we may be able to alleviate your concerns.

  2. The only disadvantage of stubs that I can think of is that they generally require more work to write than mocks, since each one of them is actually an alternative implementation of a dependent interface, so it generally has to provide a complete (or convincingly complete) implementation of the dependent interface. To give you an extreme example, if your subsystem under test invokes an RDBMS, then a mock of the RDBMS would simply respond to specific queries known to be issued by the subsystem under test, yielding predetermined sets of test data. On the other hand, an alternative implementation would be a full-blown in-memory RDBMS, possibly with the extra burden of having to emulate the quirks of the actual client-server RDBMS that you are using on production. (Luckily, we have things like HSQLDB, so we can actually kind of do that, but still, it is a bit messy, especially since it mostly sticks to the SQL standard and does not bother emulating anyone's quirks.)

  3. Good use cases for mocking are when the dependent interface is too complicated to write an alternative implementation for it, or if you are sure you will only write the mock once and never touch it again. In these cases, go ahead and use a quick and dirty mock. Consequently, good use cases for stubs (alternative implementations) is pretty much everything else. Especially if you foresee engaging in a long-term relationship with the subsystem under test, definitely go with an alternative implementation which will be nice and clean, and will require maintenance only in the event that the interface changes, instead of requiring maintenance whenever the interface changes and whenever the implementation of the subsystem under test changes.

P.S. The person you are referring to could have been me, in one of my other testing-related answers here on programmers.stackexchange.com, for example this one.

  • an alternative implementation would be a full-blown in-memory RDBMS -- You don't necessarily have to go that far with a stub. – Robert Harvey Sep 16 '15 at 19:21
  • @RobertHarvey well, with HSQLDB and H2 it is not so difficult to actually go that far. It is probably more difficult to do something half-assed in order to not go that far. But if you decide to do it on your own, you will have to begin by writing an SQL parser. Sure, you can cut some corners, but there is a lot of work. Anyway, as I said above, this is just an extreme example. – Mike Nakis Sep 17 '15 at 15:49
9
  1. The Open/Closed principle is mostly about being able to change the behavior of a class without modifying it. Therefore, injecting a mocked component dependency inside a class under test does not violate it.

  2. The problem with test doubles (mock/stub) is that you basically make arbitrary assumptions regarding how the class under test interacts with its environment. If those expectations are wrong, well, you're likely to have some problems once the code is deployed. If you can afford it, test your code within the same constraints than the one bounding your production environment. If you can't, make the least assumptions possible, and mock/stub only the peripherals of your system (database, authentication service, HTTP client, etc...).

The only valid reason why, IMHO, a double should be used, is when you need to record its interactions with the class under test, or when you need to provide fake data (which both techniques can do). Be careful however, abusing of it reflects a bad design, or a test that relies too much on the API under test implementation.

6

Note: I'm assuming you are defining Mock to mean "a class with no implementation, just something you can monitor" and Stub to be "partial mock, a.k.a. uses some of the real behavior of the implemented class", as per this Stack Overflow question.

I'm not sure why you think that the consensus is to use stubs, for example it's just the opposite in the Mockito Documentation

As usual you are going to read the partial mock warning: Object oriented programming is more less tackling complexity by dividing the complexity into separate, specific, SRPy objects. How does partial mock fit into this paradigm? Well, it just doesn't... Partial mock usually means that the complexity has been moved to a different method on the same object. In most cases, this is not the way you want to design your application.

However, there are rare cases when partial mocks come handy: dealing with code you cannot change easily (3rd party interfaces, interim refactoring of legacy code etc.) However, I wouldn't use partial mocks for new, test-driven & well-designed code.

That documentation says it better than I can. Using mocks allows you to just test that one particular class, and nothing else; if you need partial mocks to achieve the behavior you're looking for, you've probably done something wrong, are violating SRP, and so forth, and your code could stand a refactor. Mocks do not violate the open-closed principle, because they're only ever used in tests anyway, they're not real changes to that code. Usually they're generated on the fly anyway by a library like cglib.

  • 2
    From the same provided SO question (accepted answer), this is the Mock/Stub definition I was refering too: Mock objects are used to define expectations i.e: In this scenario I expect method A() to be called with such and such parameters. Mocks record and verify such expectations. Stubs, on the other hand have a different purpose: they do not record or verify expectations, but rather allow us to “replace” the behavior, state of the “fake”object in order to utilize a test scenario ... – Christopher Francisco Aug 12 '15 at 21:57
2

I think the issue may arise from the assumption that the only valid tests are those which meet the open/closed test.

It is easy to see that the only testing that should matter is that which tests the interface. However, in reality, it is often more effective to test that interface by testing the inner workings.

For example, it is nearly impossible to test any negative requirement, such as "the implementation shall not throw any exceptions." Consider a map interface being implemented with a hashmap. You want to be certain that the hashmap meets the map interface, without throwing, even when it has to rehash things (which could get dicey). You could test every combination of inputs to make sure they meet the interface requirements, but that could take longer than the heat death of the universe. Instead, you break the encapsulation a bit, and develop mocks that interact more tightly, forcing the hashmap to do exactly the rehash needed to ensure the rehashing algorithm doesn't throw.

Tl/Dr: doing it "by the book" is nice, but when push comes to shove, having a product on your boss's desk by Friday is more useful than a by-the-book test suite which takes until the heat death of the universe to confirm conformance.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.