1

I'm writing a CRUD app in Python that exposes web API. At first I wrote functions for communicating with DB and wrote tests for these functions.

def crud():
   # do something with db

def test_crud():
   crud()
   # check crud tables now have all required rows

Then I wrote functions for API routes, which do some input validation and then just call corresponding crud function. All API functions do is basically call crud functions inside them and return result. Tests for them are like:

def api():
   # do some api stuff
   return crud()

def test_api():
   resp = api()
   # check response body has what we expect
   # check crud tables now have all required rows (1)

Here at (1) it was already ringing a bell to me, but whatever.

Then I had to add a new module that does something with DB, calls some crud function and has to be used in api functions instead of crud functions.

def module():
    # do some stuff
    crud()
    # do some stuff

def api():
   # do some api stuff
   return module()

So, now tests look like this:

def test_crud():
   crud()
   # check crud tables now have all required rows

def test_module():
   module()
   # check module tables now have all required rows
   # check crud tables now have all required rows

def test_api():
   resp = api()
   # check response body has what we expect
   # check module tables now have all required rows
   # check crud tables now have all required rows (1)

What's bothering me is that I check the same thing in three places and in case of some changes I'll have to edit basically one test three times. I don't think that I should remove, for example, checks from test_api() that are related to side effects of internally called functions, because I care both about API response and these effects.

I was reading similar questions like "should I test only public methods or private too?" where, in general, answer was "test public methods, but not private ones". Does it apply to my situation, do I need to remove tests for crud and module and leave only test_api which tests all the things that the first two do?

4
  • This is an interesting question. The code snippets you give us don't really tell us anything about what you are actually testing, it appears that crud, api and module are test methods themselves, so why not directly implement them in the test_xxx methods?
    – Helena
    Nov 20, 2021 at 17:09
  • @Helena, no, crud, api and module are separate functions (actually, separate sets of functions) used in application, and test_xxx are test methods for these functions. I've added more pseudoexamples to make it clearer Nov 20, 2021 at 17:24
  • So the comments are placeholders for actual tests, and not describing the methods above?
    – Helena
    Nov 20, 2021 at 17:26
  • Yep, comments describe what is done after calling these methods Nov 20, 2021 at 17:27

3 Answers 3

3

A concept that has always stuck with me from a user group talk on testing is this:

  • Unit testing and integration testing exist on a spectrum, not either side of a neat dividing line. You wouldn't treat every function which is built into Python as a collaborator to be mocked, so even the narrowest "unit test" necessarily "integrates" lower-level code.
  • Having tests on multiple parts of that spectrum, for the same code, is a good thing. Narrow unit tests are great at proving that a component meets its designed contract - in the broad sense of what it should do, not the narrow sense of input and output types. Wider integration-oriented tests are essential for safe re-factoring, where you want to know that replacing components A and B with new components X, Y, and Z can give you the same final result.

So, the tests in your example might all be valuable, but each should be testing a particular level of abstraction, and not just asserting the same thing. For each level, consider:

  • Is the code implementing its own distinct logic, and making use of other units as collaborators, which have a particular contract? If so, substitute a mock, and test "assuming the CRUD function meets its contract, the additional logic will look like this". For instance, "assuming the database contains 5 bookings (as returned by a mock CRUD object), the module should perform this processing".
  • Is the code a higher level abstraction, making use of other units as implementation details? If so, write the test as though you don't know what those implementation details are, and consider the contract at this level. For instance, "assuming the user asks for bookings and some bookings exist, the output should be in this JSON format".
6

I think the issue here is not whether to test or not, but what you are doing with that test.

The primary purpose of Unit testing is to ensure that things behave predictably over time. If you re-write your tests every time you update your code, then it's probably not a part of your code you should be exposing as an API (internal, or external) anyway.

If you only write your tests once (or once per major revision), then any well written unit test is a valuable asset to your QA process, both for your users, and your internal developers that doesn't cost much to deploy.

Plus, you get the benefit of future-proofing your existing code if you adhere to your tests, rather than bending them to the "new" behavior.

2

What's bothering me is that I check the same thing in three places

Then you're not unit testing, you're integration testing.

A unit test tests one component, not multiple. Therefore, your module unit test should use a real module, but a mocked crud logic. Your api unit test should use a real api, but a mocked module.

You are not testing the same thing three times. You're individually testing three components that together form your business logic. The individual tests themselves are different and test different things.

The test logic would therefore be:

def test_crud():
  crud()
  # check crud tables now have all required rows

def test_module():
  module()
  # check module tables correctly display the same data that was configured in the mocked crud dependency

def test_api():
  resp = api()
  # check response body returns the same data that was configured in the mocked module dependency

I was reading similar questions like "should I test only public methods or private too?" where, in general, answer was "test public methods, but not private ones". Does it apply to my situation, do I need to remove tests for crud and module and leave only test_api which tests all the things that the first two do?

In terms of testing, "public" does not mean the same as the "public" api. It means any public behavior of a given component. If a given class has one public method, which in turn uses several private methods, you should only be testing the public method.

Your crud, module and api should be separated, as per the Single Responsibility Principle. Taking the example of a separated module class, you will then design this class so that it has the necessary public methods that the consumers (in this case the api) will use. You might use some additional private methods to further refine the internal workings of the module.

But you only write tests for the public methods of your module, since those are the methods your consumers will be using. If a private method is never called (indirectly) when all public methods have been called, then the private method is clearly not being used and therefore serve no purpose.

10
  • 1
    I agree with most of this answer, except the second paragraph. Writing integration tests as well as unit tests (and a spectrum of tests in between) is a good thing, because narrow unit tests are useless when you re-factor the application. If the high-level code is only gluing together components, then mocking those components doesn't give you a useful test - it will end up closely coupled to the implementation, and you'll end up editing the test every time it fails, rather than actually detecting defects.
    – IMSoP
    Nov 23, 2021 at 12:01
  • @IMSoP: I'm not sure where you got that I'm against integration testing, because I am not. "If the high-level code is only gluing together components, then mocking those components doesn't give you a useful test" It does give you a useful test: confirming that the components have been glued together correctly. That is precisely what happens with mocked dependencies in unit tests: confirming that the unit under test correctly handles its dependency, and not confirming a real dependency in and of itself. [..]
    – Flater
    Nov 23, 2021 at 12:19
  • @IMSoP [..] "it will end up closely coupled to the implementation" Test focus on public behavior. In cases where your unit under test is a simple wrapper, the public behavior effectively mimics the dependency's public behavior, and there is little logic to test to begin with. We can argue whether testing wrappers is meaningful or not; but this is subjective. Some want 100% code coverage even if some tests are trivial, others stick to covering only the non-trivial, others only write minimal coverage, ... There are definitely arguments pro testing wrappers, e.g. if the wrapper evolves. [..]
    – Flater
    Nov 23, 2021 at 12:23
  • @IMSoP [..] However, if a wrapper's interface evolves constantly whenever its dependency's interface does, then you've got a badly written wrapper. The point of wrapping something is to avoid making constant changes to the interface (of the wrapper) as much as you reasonably can, so that the fallout from any changes to the dependency's interface is contained within the wrapper's implementation, not its public behavior. And when the public behavior doesn't change, the tests don't change either.
    – Flater
    Nov 23, 2021 at 12:25
  • 1
    I didn't say that you are against integration testing, just that the second paragraph of this answer reads that way: "your module test should ..." implies that it "should" be a unit test. My point about mocking is that the set of dependencies a high-level component uses can change - you might split a component in two, and have to rewrite all the low-level tests, but want to confirm that the wrapper still meets its public contract. If you mocked the dependencies when you tested the wrapper, you can't run the old test with the new structure, so can't see if the refactoring broke it.
    – IMSoP
    Nov 23, 2021 at 12:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.