We're figuring out whether we should write acceptance tests that revolves around preloaded data on a database file, or programmatically added needed data per test, or filling database while testing other application features.
This is a more detailed, easy use case: you want to acceptance test (or UI-test) adding a
Book to the system. But in order to do that, you need to have
Category rows in the database already, so you can assign one to the book. As you're writing your test case for adding a book, you:
Use a previously seeded database file that has already categories in it.
- cons would be you need to keep that file up with every change in the code, with the latests changes on the system, AND you'd need to fill it while considering additional business logic (i.e. maybe you seeded authors with random parameters; but turns out an author can't be selectable unless it passes x, y and z conditions; and the seeder didn't take that in consideration)
Programmatically add the category in the test.
- cons would be you need to keep up with additional business logic, again.
Each tested feature retains data, so you first run acceptance test for adding categories, and then for adding a book.
- cons would be your testing tool would need not to isolate tests, or at least run them in a certain order.
Maybe there's a 4th option I haven't considered yet? What's the correct approach for this common task?