2

When discussing the testing approach, we had disagreements.

We develop software that we package into an image and distribute. We have two suggestions for testing:

  • Build a separate image with a test environment (dependencies for testing and test data).
  • Or build an image and mount the test directory separately.

In the second approach, there are a number of problems with dependencies for tests. And it all looks like a crutch.

Which approach would be the most appropriate for unit and integration testing?

Should the distributed image be tested at a higher level during system testing?

It seems that unit testing should not test the image itself. On the other hand, testing one image and distributing another looks strange.

What do you think is the right way?

6
  • Proponents of testing pyramid say you should do both. (Many low level intrusive unit tests, a few high level black-box system tests). I'm not one of them though.
    – Basilevs
    Commented Jun 28 at 7:45
  • 1
    Duplicate of Do I need unit test if I already have integration test?
    – Basilevs
    Commented Jun 28 at 7:50
  • 2
    @Basilevs It was about something else. I didn't say that we don't need any tests. I'm talking about the environment. Is it worth building two images (1 for unit tests with appropriate dependencies (for example, if it's python, then pytest, mypy, etc.) and 1 for production (without these dependencies)). Anyone should build an image and somehow mount a directory with tests to it.
    – Vladimir
    Commented Jun 28 at 9:09
  • 2
    For what purpose are you using this containerized environment? If you want to test the image that you'll deploy, testing a different image won't help. But if your problem is that devs are using very diverse systems and you want to use a container as a canonical test environment, then installing development tools into that environment sounds reasonable (though in case of Python, using venvs is usually sufficient isolation). Mounting test files and source code (not test tools) into a container can still be helpful to speed up the write-compile-test cycle.
    – amon
    Commented Jun 28 at 9:26
  • 1
    @Vladimir you DO need to test the exact deliverable, therefore the question is whether you need another image with injections. And that question is exactly about that.
    – Basilevs
    Commented Jun 28 at 10:11

1 Answer 1

9

Different kinds of tests are for different audiences.

Those different audiences have different needs. The tests don't care where they are run. They care about the API they are run against and what the subject of the test does when tested. The users of the tests care about how long they take.

A low level unit test that proves to developers that we've implemented a Fast Fourier Transform correctly doesn't care about your images. Just give it access to the FFT API and let it run.

However, a sound wave visualization acceptance test, that the product owner wants to be rock solid sure is part of this deliverable, that maps directly to a feature, ya know, a requirement, well yeah, it would be justified to test that against the distributed image, regardless of what the unit test does.

Does that mean if the visualization uses the FFT we don't need the unit test? No. It doesn't. Stop trying to ignore the needs of developers. We need testing at all these levels for all these people. The fact that the tests are redundant isn't the point. It's the needs of the people using the tests that makes this different.

So my answer is support both.

Imagine walking onto a movie set and trying to take away a camera mans light level meter because, "he can just watch the finished movie". Sigh. Please, stop focusing on the technical truth here and think about what you're doing to the people.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.