2

I am talking about these two testing strategies :

  1. Have a pyramid of tests (with many more unit tests than high-level tests), because high level tests are harder to maintain and slower (see also : https://blog.octo.com/the-test-pyramid-in-practice-1-5/)

vs

  1. Test behaviors (high level tests), not implementation details so that refactors don't break your tests when initial input and final output and general behavior are the same.

For me, it looks like those concepts are contradictory.

Here is a concrete example : let's say I have 2 local models, DTOs (close to API) and entity model (close to database structure).

Now I have mappers dto <-> entity automatically generated (using MapStruct, Orika, ...), if I rename an attribute in one of the model (even using automatic IDE refactor), it will still compile and I will still have the same input/output objects but it will break my system details (the attribute won't be mapped through the generated mapper), so it seems like it's useful to write a unit tests for each mapper (which would be very tedious and defeat the purpose of having the mappers auto-generated in the first place).

On the other side, if I write only integration tests going through all my layers, I'm not respecting the "pyramid of tests" recommended strategy.

Improve this question
16
  • 2
    "High-level" in the test pyramid sense and "high-level" in behavioral tests sense are not the same. One is about the scope of what's under test, the other is about how you express, in code, the rules and relationships pertaining to the component being tested (i.e. is the test explicitly expressing a rule or a constraint, or are you implicitly testing the rule by relying on specific values and implementation details).
    – Filip Milovanović
    May 29 at 7:25
  • 2
    The entire point of unit tests is to test behaviour, not implementation. Even something notionally as simple as multiplication could be implemented in totally different ways, and the test shouldn't care.
    – Kilian Foth
    May 29 at 8:24
  • 1
    Hi Doc, my question is precisely about the "guideline" and about this mapper example which is very common (maybe 80% of apps ?). How can I rephrase it so it fits your standards ?
    – Tristan
    May 29 at 9:22
  • 2
    I tried to answer several times and every time I tried I realised how many things the OP was conflating or assuming. I think, @Tristan, you should read more about this topic (Pyramid) to realise that it has nothing to do with how many tests (of one type or another) you write. It's not even a best practice or principle. It's not a technique either. It's a strategy to balance the cost/time of systems testing. The pyramid illustrates how most of the confidence (and savings) comes from tests cheap to write, easy to change, fast to execute, but overall, deterministic.
    – Laiv
    May 29 at 13:52
  • 1
    Really sorry to see this closed. I think this is a good challenging question. It had me contemplating how much of the test pyramid is based in structural thinking. Pity it went this way.
    – candied_orange
    May 30 at 5:36

2 Answers 2

Reset to default
4

I think things start to make sense once you concede that "unit test" should not test a unit of code (like class or a function) but unit of behavior.

Imagine your application has simple Create/Get behavior. Then, you unit test would look like this:

  1. Create an item in system, receiving Id
  2. Get an item from system by Id
  3. Assert retrieved item is same as created item

This test tests unit of behavior in your system. This test can be made fast and isolated by having an in-memory database/repository implementation and creating instances of necessary classes just for this specific test. Thus putting it in bottom part of the testing pyramid, satisfying #1. But it also deals with high-level behavior of your system, satisfying #2.

When test is defined like this, if and how things are mapped inside the system is an implementation detail. The test doesn't care if you have manual or automated mapping. Or how the item is stored in database. This test will of course change if you change the item's structure, but that is expected as that is externally visible change to the system.

Improve this answer
4
  • Ok, so when u have a layered web app (controllers, services, repositories, mappers, ...) you should only test end-to-end use case visible from API or UI users and never individual components ?
    – Tristan
    May 29 at 10:35
  • 1
    @Tristan Forget anything about how you would structure/layer a web app. Test-drive your design as above. Start with your whole service as single "module" that has tests written against it's behavior. Create abstractions for external dependencies and create fake implementations for tests. At some point, you might realize there is "submodule" growing inside your module. At that point, it would make sense to separate the two and start writing tests against this new module. This way, your design will be perfet fit for your tests, not other way around, which is what you are trying to do here.
    – Euphoric
    May 29 at 10:41
  • I disagree with this answer's example being a good unit test. I agree that a "unit" is not necessarily a single class, but steps 1 and 2 are individual units of behavior in and of themselves. Combining them proves nothing more than testing them separately would, but it adds more pressure to the mock being consistent between the two individual steps. Ideally your mock would be able to maintain state, but this added complexity in the test is irrelevant as to the test suite's real purpose. If the only added value is that it verifies the mock's implementation, it's not a relevant thing to do.
    – Flater
    May 30 at 3:00
  • 1
    @Flater In my defense, that is how Abstract Data Structures are defined in Computer Science. Reading, en.wikipedia.org/wiki/Abstract_data_type , you will notice "invariants" of data structures are defined exactly how I define a good "behavior" test : What sequence of operations is allowed on a state V.
    – Euphoric
    May 30 at 3:47
0

The test pyramid is not a universal rule, it very much depend on the particular application. For complex logic like a compiler, you would probably have lots of unit tests. For a CRUD-style web app integrating multiple data sources, there might not be a lot of complexity in any individual module, but the challenge is integrating everything correctly end to end, so there is more value in focusing in integration tests.

Your example of mapping data correctly through multiple layers is the kind of concern best tested through integration tests.

Improve this answer
9
  • by "integration" u mean testing multiple layers/components at once, or testing the interaction of components like testing the mapper ?
    – Tristan
    May 29 at 8:42
  • @Tristan you should not test autogenerated code, for the same reasons you should not test frameworks and libraries. The way to "test" these is by writing tests oriented to prove that everything is "glued" and working as expected, and you do so by inspecting and checking the end of the integration (result). Mappings are tested "implicitly" not "explicitly" unless you write the code of the mappers
    – Laiv
    May 29 at 8:45
  • @Laiv : I'm not sure to understand : wether I write test for the mappers or not, they can either way fail for renaming an attribute (well no if I write them, IDE refactor will impact my written mappers, so I dont even need tests in this case).
    – Tristan
    May 29 at 9:28
  • Tests are not written to capture points of failure, but to build confidence and certainty around code you write. Autogenerated code is not code you write. The code you don't write is code you integrate with, hence it's tested via integration tests or E2E. In both cases, you test mappings "implicitly" by inspecting outputs/side effects, not by testing intermediate stages of the execution path. In other words, you know mappings work, because the endpoint response was right in both: format and content.
    – Laiv
    May 29 at 11:13
  • 2
    If your application is all about CRUDs and DB to DTO mappings with no business logic in the middle, there's no unit test to write. Only integration tests. That's fine, correct and comes to say, the practical pyramid of tests doesn't apply to you.
    – Laiv
    May 29 at 11:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.