4

I am writing a unit test with mocks and I am concerned about over specifying the test. The idea being that if the implementation changes in a compatible way the test shouldn't fail since that makes the test fragile and inconvenient.

For example, if the implementation calls mock.add("foo") followed by mock.add("bar") or it calls mock.add("foo","bar") it shouldn't matter, but from a mocking perspective you are then lead to validate all the possibilities that could have been implemented. For methods with lots of overloads for convenience (eg System.out.println) this would be exhausting!

So, to be clear, my question is: how to work with mocks when they are many equivalent methods that could be called by the implementation in such a way that the resulting tests aren't fragile?


PS- Not that I think it matters, but I am on Java using Mockito, but I feel there is some cross-language-applicable approach I'm missing. Our current approach is to just make fragile tests and fix them later.

  • Is your problem the tests, or is it the API design? One of the purposes of unit testing is to expose those areas in your API that are difficult to test. – Robert Harvey Oct 6 '16 at 18:04
  • The tests. If you are calling a method that is heavily overridden (eg print methods often have a form for each primitive and then objects) or some sort of builder then you have many equivalent interactions. I mean print("h");print("i"); is the same as print("hi"), so the question is about how do you handle multiple equivalent implementations in a non-fragile and non-exhaustive (ie covering every case apriori) way – ArtB Oct 6 '16 at 18:08
7

My opinion is, don't test that the implementation matches expectations. Test that the result matches expectations. To that end, you should not be expecting two calls to an add() method on your mock during your test run. You should instead expect, as a result of the test run, that "foo" and "bar" are now elements in a simple list, into which the mocked implementation(s) of add() will place any passed parameter values. You can do this with a hardcoded mock implementation, and most mocking frameworks also allow for a simple implementation of a mocked member, in the form of a lambda statement or other delegate.

  • Hmm, in my case their is no result because the result is built by the mock. I am testing a driver that calls a builder. The builder has many complex dependencies like a database connection, so its mocked out. So I'm try to verify if the driver is building a correct object from it's inputs (ie a input file). And of course builder objects often have lots of convenience overloads, and thus my problem. – ArtB Oct 6 '16 at 18:16
  • 5
    Right. So, instead of verifying that you called a very specific and thus brittle set of methods as part of your test, verify that whatever was called would have put the builder in the correct configuration to generate the correct object. You do this by setting up your mocked methods to do something simpler and side-effect free that nevertheless allows you to discern what happened, like logging simple entries in a string, StringBuilder or some collection type. Then you check what was done and verify everything that needed to happen, however it was actually done, did happen. – KeithS Oct 6 '16 at 18:24
0

I would attempt to factor my code in such a way that I have classes with

  • lots of logic, not many dependencies
  • many dependencies but not much logic (coordination logic only)

I seem to experience problems similar to yours when I have classes where I intermingle coordination with actual important logic. Sometimes you can avoid it, sometimes you cannot easily, but good to keep in mind that this option exists.

Unit testing coordination logic is extremely cumbersome and fragile. With mocking dependencies you really are only setting up your state to get down to that very important logic you are actually trying to test. If you attempt to separate the two it becomes easier to test, because you can unit test the class with 'lots of logic not many dependencies' while integration test the class that is mostly concerned with coordination.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.