Should I unit test functions internally used by API I expose?

Question

I'm writing a CRUD app in Python that exposes web API. At first I wrote functions for communicating with DB and wrote tests for these functions.

def crud():
   # do something with db

def test_crud():
   crud()
   # check crud tables now have all required rows

Then I wrote functions for API routes, which do some input validation and then just call corresponding crud function. All API functions do is basically call crud functions inside them and return result. Tests for them are like:

def api():
   # do some api stuff
   return crud()

def test_api():
   resp = api()
   # check response body has what we expect
   # check crud tables now have all required rows (1)

Here at (1) it was already ringing a bell to me, but whatever.

Then I had to add a new module that does something with DB, calls some crud function and has to be used in api functions instead of crud functions.

def module():
    # do some stuff
    crud()
    # do some stuff

def api():
   # do some api stuff
   return module()

So, now tests look like this:

def test_crud():
   crud()
   # check crud tables now have all required rows

def test_module():
   module()
   # check module tables now have all required rows
   # check crud tables now have all required rows

def test_api():
   resp = api()
   # check response body has what we expect
   # check module tables now have all required rows
   # check crud tables now have all required rows (1)

What's bothering me is that I check the same thing in three places and in case of some changes I'll have to edit basically one test three times. I don't think that I should remove, for example, checks from test_api() that are related to side effects of internally called functions, because I care both about API response and these effects.

I was reading similar questions like "should I test only public methods or private too?" where, in general, answer was "test public methods, but not private ones". Does it apply to my situation, do I need to remove tests for crud and module and leave only test_api which tests all the things that the first two do?

Answer 1

A concept that has always stuck with me from a user group talk on testing is this:

• Unit testing and integration testing exist on a spectrum, not either side of a neat dividing line. You wouldn't treat every function which is built into Python as a collaborator to be mocked, so even the narrowest "unit test" necessarily "integrates" lower-level code.
• Having tests on multiple parts of that spectrum, for the same code, is a good thing. Narrow unit tests are great at proving that a component meets its designed contract - in the broad sense of what it should do, not the narrow sense of input and output types. Wider integration-oriented tests are essential for safe re-factoring, where you want to know that replacing components A and B with new components X, Y, and Z can give you the same final result.

So, the tests in your example might all be valuable, but each should be testing a particular level of abstraction, and not just asserting the same thing. For each level, consider:

• Is the code implementing its own distinct logic, and making use of other units as collaborators, which have a particular contract? If so, substitute a mock, and test "assuming the CRUD function meets its contract, the additional logic will look like this". For instance, "assuming the database contains 5 bookings (as returned by a mock CRUD object), the module should perform this processing".
• Is the code a higher level abstraction, making use of other units as implementation details? If so, write the test as though you don't know what those implementation details are, and consider the contract at this level. For instance, "assuming the user asks for bookings and some bookings exist, the output should be in this JSON format".

Answer 2

I think the issue here is not whether to test or not, but what you are doing with that test.

The primary purpose of Unit testing is to ensure that things behave predictably over time. If you re-write your tests every time you update your code, then it's probably not a part of your code you should be exposing as an API (internal, or external) anyway.

If you only write your tests once (or once per major revision), then any well written unit test is a valuable asset to your QA process, both for your users, and your internal developers that doesn't cost much to deploy.

Plus, you get the benefit of future-proofing your existing code if you adhere to your tests, rather than bending them to the "new" behavior.

Answer 3

What's bothering me is that I check the same thing in three places

Then you're not unit testing, you're integration testing.

A unit test tests one component, not multiple. Therefore, your module unit test should use a real module, but a mocked crud logic. Your api unit test should use a real api, but a mocked module.

You are not testing the same thing three times. You're individually testing three components that together form your business logic. The individual tests themselves are different and test different things.

The test logic would therefore be:

def test_crud():
  crud()
  # check crud tables now have all required rows

def test_module():
  module()
  # check module tables correctly display the same data that was configured in the mocked crud dependency

def test_api():
  resp = api()
  # check response body returns the same data that was configured in the mocked module dependency

I was reading similar questions like "should I test only public methods or private too?" where, in general, answer was "test public methods, but not private ones". Does it apply to my situation, do I need to remove tests for crud and module and leave only test_api which tests all the things that the first two do?

In terms of testing, "public" does not mean the same as the "public" api. It means any public behavior of a given component. If a given class has one public method, which in turn uses several private methods, you should only be testing the public method.

Your crud, module and api should be separated, as per the Single Responsibility Principle. Taking the example of a separated module class, you will then design this class so that it has the necessary public methods that the consumers (in this case the api) will use. You might use some additional private methods to further refine the internal workings of the module.

But you only write tests for the public methods of your module, since those are the methods your consumers will be using. If a private method is never called (indirectly) when all public methods have been called, then the private method is clearly not being used and therefore serve no purpose.