1

We are trying to work out the best way to setup test data when we run our isolated functional tests. These tests test just the service and mock out other dependencies and use an in memory database. Each test will require some data to be setup and there are two thoughts about how this setup is done:

Inject the data directly into the database. The complaint here is that this is brittle and ties the tests to the structure of the database.

The alternative proposed is to use the services API to insert data as part of the test setup. The issue with this is that the scope of test has expanded to include the logic of inserting the data as well as the actual test case itself.

Personally I favour injecting into the database but as there is no clear consensus here I'd like to see what others think!

  • 2
    see my answer, which is basically, dont use an in-memory db softwareengineering.stackexchange.com/questions/365575/… – Ewan Aug 31 '18 at 13:47
  • That doesn't look very isolated. – Goyo Aug 31 '18 at 13:50
  • Of how many entities and relations in the database are we talking about in this question ? Isn't the insert logic subject of some tests so that you could rely on it for further tests ? If not, is there a reason; do you have architectural constraints that prevents this decoupling ? – Christophe Aug 31 '18 at 18:49
1

In short, all other things being equal, injecting the data into the database is the better approach. It reduces tight coupling between the separate business logic features. It also

I've struggled with the same question not too long ago. For reference, I finally convinved the PM to let me implement a basic amount of unit testing for a project that has been developed with no testability in mind.

The data in our project is shaped by a series of operations - the project is an application which performs these many operations. However, there is so much generated (meta)data in after the first X steps, that it became nigh impossible for me to generate the actual data state I needed to test (and in some cases there were 100+ combinations of data sets to test).
This was further compounded by the fact that the data structure was continually changing because of a bad and vague analysis and an equally vague project owner.

There are a lot of flaws in the current project management. The analysis should not have been vague. The code needed to be developed with testability in mind. The entire data structure should have been revisited instead of copy/pasted from the last similar project.

Nonetheless, this is the project I was writing unit tests for, so I had to face the facts. There were ulterior reasons (not directly related to my development) which made the proper approach (data injection) nigh impossible to get right.

At this point, I decided to therefore rely on the business logic to arrange the state of the database.
Am I happy about it? No. It feels so very dirty. However, I do have to admit that it does actually work given the less-than-perfect situation the project already finds itself in.

The biggest drawback here is that a small error in one of the first operations will cause many tests to fail, since almost all subsequent operation logic depends on that first operation to be correct.
But due to the linear nature of the operations, it's still relatively easy to spot which test is the cause. We actually ended up numbering our tests so that the order of operations (and thus the order of tests depending on each other) became obvious.

By default, I do not recommend choosing to rely on the business logic to set your data state. However, there are some situations where the data is so complicated and you're unable to actually change that fact. In those situations, it may be beneficial to simply have the logic generate the data state for you simply to ensure that you can have some tests instead of none at all.

  • Another benefit to inserting data in the database while bypassing business logic is for tests against old data being used with new business logic. The current state of your business logic is not necessarily the same as when old data was inserted. Bypassing business logic allows you to set up these "old data with new logic" tests. – Greg Burghardt Aug 31 '18 at 16:48
0

It all depends on the kind of tests you are intending to perform.

  • For unit tests of the data-access/data-source layer, data injected directly in the database is the best approach. With precise control on the test data, you reduce the potential cause of errors to the tested object itself. In addition, you can verify that your data-source behaves as expected with incomplete or inconsistent data.

  • For any other tests, i.e. unit tests of the business logic, or integration tests:

    • db injection could still be a valid approach, but if and only if the data model is simple.
    • on the contrary, db injection could be counter productive with a complex data model, because you introduce the risk of unexpected inconsistencies in the test data that could be falsely interpreted as error in the tested component.

So for complex data models (and data-source tests excepted), the better approach would be to create test data using the API. This works very well if using the following mitigation measures:

  • ensure that the data creation API is designed in a way to facilitate test automation (for instance, the onion architecture sees the testing components as integral part of the architecture)
  • make sure that the unit tests of the writing API is unit tested first, so that the writing errors could be isolated and would not pollute the dependent tests
  • if the data is very long to be produced, the you could imagine a mixed approach: create the heavy test data once. Create a snapshot of the database. Restart tests from the snapshot (i.e. it's as if you would inject data, but without the risk of accidental inconsistencies). continue the remaining tests using API if necessary.

This gives you the best of both worlds (even if you'd be developing an ERP systems with thousands of interrelated tables and tricky binary data types).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.