0

I have a similaire scenario:

int retrievePrice(string productName) {
   string id = restApi.get("productid", productName);
   return restApi.get("productprice", id);
}

The restApi could also be a third party. In order to make the test self sufficient, I end up mocking the restApi.

If I directly test this method, I will end up testing the mock and my test will be coupled with the production code. Thus, any refactor will be painful.

Should I only test a higher level function? If it is the case, should I mock restApi or the whole retrievePrice function? It is ok to leave this method untested?

Edit:

  • The function under test doesn't have a side effect, thus I don't need a spy to test the logic of this function.
  • Outside of the short/mid term benefice of having tests, I am more concerned about the long term. The day when I will need to change the implementation (without breaking the contact), I expect my tests to still pass. Maybe it is a mocking framework problem (mock on demand vs fully mock a class).
Improve this question
4
  • 1
    Does this answer your question? Is there any benefit testing only with mocks/fakes/doubles?
    – Greg Burghardt
    Oct 7, 2021 at 17:46
  • 2
    "I will end up testing the mock and my test will be coupled with the production code" - that's a very strange sounding sentence, don't you think? :D If your dependence on a mock means that you're coupled to production code, than that must mean that your tests and/or mocks are coupled to the implementation details of production code - which is not supposed to be the case. The whole point of a mock is that you can change the thing it mocks, and not change the mock (or change it rarely).
    – Filip Milovanović
    Oct 7, 2021 at 18:53
  • George Z. is testing a side effect and he want to validate that it really happen. In my case, I have the result in the return value. The behavior is validate with the result of the function not the steps inside it. Tomorrow I could have a new endpoint "productprice" which take a product name and the function will have the same behavior, thus I don't expect my test (happy path) to fail
    – Olivier D
    Oct 7, 2021 at 19:04
  • 1
    I might not write a test for such a function because it contains no real logic, but there are things that could fail, e.g. a typo in the endpoint for the REST API. But that's not the kind of bug you're likely to detect via a mock. From a black-box perspective, there might also be logical questions like “what should the function return if the product doesn't exist” – it's not obvious that an exception should bubble up, perhaps it should return an int?. That said, if you already have a way to easily test against a mock API, adding the relevant tests here would be simple and probably worth it.
    – amon
    Oct 7, 2021 at 19:14

3 Answers 3

Reset to default
6

It helps to think this through in the other direction.

You're not getting the right output. You don't know why. From here, before you've looked any further, would it be helpful to you to know whether the bug is located in retrievePrices or in restApi?

If you agree, then you should confirm whether retrievePrices actually works with its restApi dependency correctly. If it is doing it correctly, then the bug is clearly located in restApi itself and you don't even need to look in retrievePrices while hunting for the bug. That can help save you some time and effort.

This is the value of your unit test: to know if your unit handles its (mocked) dependency correctly, so that you can pinpoint the component in your codebase that is failing to uphold its duties.

Sometimes, unit tests feel almost too simple when writing them, but that does not always mean that the test does not add value.

Write tests when they add value, not because they're not simple.

Improve this answer
4
  • There is a lot a good points, but what about the maintenance/coupling of the test + mock with the production code? Maybe I am over thinking this, but I expect my unit test to pass independently of the implementation. Maybe it is also a mock framework problem, there are so easy to use but I end up only mocking the required method, not the whole class, thus breaking multiple tests if the implementation changes.
    – Olivier D
    Oct 8, 2021 at 12:59
  • @OlivierD " expect my unit test to pass independently of the implementation". Then what is your unit test testing, if not the implementation?
    – Flater
    Oct 8, 2021 at 22:23
  • I test the behavior and I will now also look for the contact. When I call a function I shouldn't care how is it implemented. retrievePrice can call 1 or multiples endpoint, use a cache, but at the end of the day in a test perspective, I only care about the result.
    – Olivier D
    Oct 14, 2021 at 17:11
  • @OlivierD: I think you've struck on a nuanced difference here. The test does not directly interact with an implementation, nor does it know what the specific implementation is. However, this is done specifically so the test is an impartial arbiter who is reasonably able to decide that the implementation is worked as specified by the contract. The test is there to vet the implementation.
    – Flater
    Oct 15, 2021 at 9:17
5

There seem to me to be at least three useful tests you can perform here:

  • The happy path: both API calls return successfully. You are testing that the id returned from the first call is passed to the second call, and the function returns the correct value from the second call.
  • Unhappy path 1: the first get call fails (in the real world, you could have multiple tests here for all the different ways the call could fail).
  • Unhappy path 2: the second get call fails (ditto)

All of those seem potentially useful tests. It's up to you to work out if writing them is the best use of your time.

Improve this answer
10
  • I don't think I should test unhappy path, I should let the exception bubble up, unless I have a smart way to handle error, which is more that often not the case. Also I don't think there is a solid way to test all unhappy path. The lack / wrong documentation or the third party changing overtime make it impossible to relie on mock.
    – Olivier D
    Oct 7, 2021 at 18:47
  • 5
    "Throws exception if the API calls throws an exception" is still something you should test.
    – Philip Kendall
    Oct 7, 2021 at 19:10
  • In this case am I testing "retrievePrice", "restApi.get", the lack of try-catch or the programming language it self?
    – Olivier D
    Oct 7, 2021 at 19:34
  • 2
    @OlivierD See my comment on your question. You could test for the (absence of the) return value of retrievePrice(). It is not necessarily obvious that restApi.get() should raise an exception if the resource doesn't exist and that this exception should bubble up. But in the big picture, this test seems less important than others.
    – amon
    Oct 7, 2021 at 19:48
  • 2
    @OlivierD You are testing your contract, which includes "can throw an exception". I appreciate this is about to run headlong into the whole checked vs unchecked exceptions debate.
    – Philip Kendall
    Oct 7, 2021 at 20:20
0

The restApi could also be a third party. In order to make the test self sufficient, I end up mocking the restApi.

This sounds reasonable

If I directly test this method, I will end up testing the mock and my test will be coupled with the production code. Thus, any refactor will be painful.

The test might need to be changed if the restApi is changed, but backward-incompatible changes to api's should be a fairly rare occurrence, and would affect much more than unit-tests. And I do not see how simply changing the implementation of the method would affect the test. It would presumably still need the same data from the Api, and still maintain the same contract to test against.

Should I only test a higher level function? If it is the case, should I mock restApi or the whole retrievePrice function? It is ok to leave this method untested?

I'm of the opinion that unit tests should bring a overall positive value to your business. But there are many factors that can affect that. How expensive would a bug be? How expensive is it to write and maintain a test? How likely is it that a bug would be caught by the unit test? Each factor affect the value proposition of a test.

I personally find trivial methods least useful to unit test, complex mathematical methods most useful, and methods that mostly integrate other methods, like this example, somewhere in the middle.

Also consider that unit tests are not the only type of automated test. It might also be useful to test larger sections of code to check that they work correctly together, and this might include tests that uses a test-version of the actual API to ensure that it remains consistent.

So my recommendation would be to write the tests that you think would be the most useful. This might not be easy to decide, so consider asking colleagues that know more about the context, and consider that you can always add or remove tests at any time.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.