How can we better maintain a large amount of DbUnit datasets?

Question

The project

I'm currently working on a Spring Boot project, and I have a question regarding testing against the database. Our testing tech is currently JUnit 4, Mockito and DbUnit, and we test queries against an in-memory H2-database using JdbcTemplates. The project is barely 2 months old, and we already have between 30-40 tests that are annotated with some combination of files that specify the initial state of the test, @DatatabaseSetup('state-prior-to-test.xml') and the expected state after our test, @ExpectedDatabase('expected-state-after-test.xml'). The tests are quickly growing unmaintainable, and I would like some insight or alternative solutions to our approach. Below is a quick rundown of how we might do a DbUnit-test.

---

An example test

UserDaoTest.java:

@Test
@DatabaseSetup("setup.xml")
@ExpectedDatabase("expected.xml")
public void userRename_whereUserExists_shouldRenameUser() {
    userDao.renameUser(1, "Manning");
}

setup.xml:

<?xml version="1.0" encoding="UTF-8"?>
<dataset>
    <user id="1" name="John"/>
</dataset>

expected.xml:

<?xml version="1.0" encoding="UTF-8"?>
<dataset>
    <user id="1" name="Manning"/>
</dataset>

The test goes like this

1. First, we setup the initial state using the @DatabaseSetup-annotation. This creates the user-record in the test-database (in our case, an in-memory H2-database)
2. Then, we attempt to rename John to Manning using the renameUser-method of the UserDao-class
3. Finally, we assert that the name of the user-record with id = 1 is now Manning with the @ExpectedDatabase-annotation. This looks at the resulting data in the user-table of the test-database

This way of asserting database state has been useful and I won't deny that it provides value, but keeping our setup- and expected-states specified in XML files is already proving to be hard to maintain, and I am anticipating it to become a maintenance nightmare too soon.

---

Maintenance challenges

Refactoring is tough
Using refactoring tools such as renames and extractions are oblivious to the structure of the code in relation to the elements of the xml files. A great benefit from having tests is being able to make a tiny change, then run the tests ensure that change did no harm.

For example, let's say we wanted to rename the name-property of the User-object to firstName. Using refactoring tools would allow us to make this change in our entire project in seconds. But what about the leftover setup/expected xml-files? Refactoring tools are oblivious to the contents of these files, and as far as I can see, we are left with 2 options: manually updating every xml-file (after running all tests and locating them), or search/replacing the attributes of the user-elements. This is clearly barking up the wrong tree.

File organization
We occasionally share setup-files between tests, but we never share expected-files. This means that for 30 tests there are probably around 45 xml-files. Not only is this a lot of files to maintain, but they have to be organized somehow. Regardless of where you put them, you have to specify their location as string-parameters to the annotations specified above, like @DatabaseSetup("../database/userdao/setup/user-setup.xml"). These string-parameters also have to be maintained. They also give us no way of navigating to the file using any cleverness of the IDE, so it's all very manual and rigid.

---

The question

Given the current state and trends of agile and DevOps, developers are expected to be very flexible to change. This way of testing is making us the complete opposite.

What are some alternative strategies or technologies that can help us test the behaviour of database-operations in dao-tests or larger integration-tests while at the same time remaining flexible to change?

Thanks in advance.

Answer 1

You don't want to use external files to describe internal objects whose interface might change often. Either code the assertions directly in JUnit (so that they are refactoring-friendly) or use a more stable interface in your dataset files (such as the database schema).

In your example, "where user exists ... should rename user" seems simple enough to be fully described in code. Dataset files are most useful when you are testing a complex end-to-end use case where it is inconvenient to put all the setup and expectations in your code, however in these cases there is already a well-defined interface that you can use (the public interface of your application). You will also have fewer files to manage if you only use them for end-to-end tests, which should help with the file organization problem.