1

I have looked through most of the answers given regarding using in-memory database for unit test, but I could not find one that was clear to me.

I am writing some unit tests to cover some of our codebase. The codebase is written in ASP.NET Core and EF Core. I am using xUnit and Moq.

I have read and understood the concept of unit test vs integration tests.

As I understand, writing unit tests means testing a code in isolation from other dependencies and to achieve this, we can mock those dependencies so we can test the code only.

However, I found that it is a bit more work setting up mock of dependencies that I need especially if these dependencies are repository dependencies.

When I tried using an in-memory database, all I need to do was setup the in-memory database, seed the database, create a test fixture and it works. And in subsequent test, all I have to do is create the dependencies and it works just fine.

To mock the repository, I have to setup a mock and return value. And then there is the complexity of mocking an async repository as explained here: How to mock an async repository with Entity Framework Core. For each test, I have to mock the repository that is needed which means more work.

With all these in mind, would it be better if I just ditch mocking and use in-memory database? Also, would this not be seen as integration testing though it's an in-memory database.

I created two different tests using the in-memory database and mocking.. - The in-memory database tests understandably took more time, but the difference is usually about 1sec longer than the tests using mocks.

8

IMHO you are asking the wrong question. It does not matter if what you create is called a unit test by some people, or an integration test by others. What matters is here,

  • is this test useful for your case (so will it help you to avoid certain defects, and will it suffciently reduce the area in code where the root cause of a certain defect may be)?

  • is it fast enough, even when you are going to run several tests of this kind? (1sec longer does not sound much, but when the original test required 0.001 seconds, and the new one requires 1.001 seconds, running 5000 tests of this kind will make a notable difference)

  • is it maintainable? Ideally more maintainable than alternative approaches? This depends heavily on the tooling and how good an "in-memory DB" is supported.

If you can answer all of these questions honestly with "yes", then go ahead.

Recommended: The Way of Testivus - which tells you, for example, to follow less dogma (which is a good recommendation not just for testing).

  • thanks a lot for the detailed answer. I can honestly answer yes to point 1 and 3. For point, it is worth noting that the setup for the in-memory database occurs once as this is a shared context so the same setup is used across all tests. I wrote two tests using the same in-memory database and though the first took about 1.5s, the second took 74ms. The second test for using the mocking method took about 20ms compared to about 350ms for the first. These numbers are average of course. Thanks for the link too – elfico Mar 26 '20 at 13:06
  • 3
    @elfico: this can work well, but be careful: when reusing the same db for several tests (in-memory or not), one has to make sure not to get unwanted side effects. It can also prevent parallelization of tests. It depends all on your specific requirements. – Doc Brown Mar 26 '20 at 13:21
  • 1
    @elfico: Sharing a database between tests can create a coupling, where for example test 5 will start failing if tests 2 and 4 are not executed before it in that order. That limits your ability to run a subset of the tests. – Bart van Ingen Schenau Mar 27 '20 at 6:37
4

An in-memory database can be useful for both unit tests and integration tests, but it depends on what precisely you are trying to do.

Unit tests check a single component. Ideally that unit is tested in isolation from other components, but that's not strictly necessary – using other tested components in a unit test is OK as a matter of convenience. Isolated tests tend to run faster and point closer to the cause of an error, but can be difficult to set up unless the software has been designed for testability, for example by minimizing dependencies between components and connecting components over small, easy to mock interfaces.

Integration tests check the interactions between components. However, components that are irrelevant in the context of that test can still be mocked out. But again, that's a tradeoff between speed and error localization versus convenience. Some tests are so difficult to set up that they are not worth setting up!

An in-memory database is useful for both unit tests and integration tests when you don't want to mock out a complete data access layer, or if you need to have a real database due to an ORM. Here, using an in-memory database is easier to set up, is faster, and can easily provide isolation between tests by setting up a new database for each test.

However, this won't test the integration between your software and the actual database – these differences can be significant due to SQL dialects etc. Therefore, a test plan should also include tests with the real database sofware, e.g. in a staging environment.

In practice, keeping a clear distinction between unit tests and integration tests is not overly important. It is more important that you have a good automated test suite, and perhaps tooling and instructions for reproducible manual tests. Personally, I'm mostly writing integration tests because they make it easier to demonstrate the value of the software under test, and because they provide better confidence that the software works as a whole. I've found BDD-style integration tests to be particularly helpful, although they still assume the use of fine-grained TDD-style unit tests.

  • 2
    "In practice, keeping a clear distinction between unit tests and integration tests is not overly important." I agree from the perspective of the textbook testing responsibility of a developer; but there are other considerations here. Integration tests often take longer than unit tests (depending on how rigorous they are and how many cases they test), which for larger projects can be an issue for developers to run locally every so often. It's perfectly acceptable to split the tests so devs run the unit tests and the build pipeline runs unit+integration tests. – Flater Mar 27 '20 at 8:54
  • 1
    @Flater Absolutely! I've previously argued that point as well, but think that “Separating automated integration tests from unit tests is not in itself important, it is just a performance hack to improve your test feedback”. But in this answer, I'm trying to point OP towards doing things that work, without overthinking the official definitions of unit and integration tests. – amon Mar 27 '20 at 9:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.