3

We are having problems in testing routes that modify the data in our API. For example the test that deletes a resource will fail if run the second time.

Also due to business logic you can't create or delete some resources via the API (as they are created by other triggers, or as a user you can't delete data but only archive).

Also you can't create resources with a particular id, it's generated by the API, so it can be hard to test some special situations and not all test suits permit getting the data from other requests to reuse later on. And this would also break running tests independently as it will create dependency between tests.

Also redeploying the whole application for one test is not ideal, as it will slow down the whole development of our application.

So what would be the best approach for re-seeding the db? Was thinking to add a route like:

DELETE /all/changes/to/just/test/user/

But that breaks the responsibility principles, that the API shouldn't be responsible for testing and that this should be the job of the tests.

Update:

The API/web app has a complicated stack based on more docker microservices, events queued in rabbitMQ, elasticsearch, mongodb, external file storage, third party authentification, docker, kubernets, horizontal scalability, etc.

So it's not just a DB to mock, and would like to have the testing environment as close to production as possible. Also I would like to be able to run and debug the tests from the local env, as I'm creating them. As having to deploy the whole stack again just to run and debug one test would a massive overkill and would slow down development a lot. Also, as a nice to have, we would like to use ATDD.

4

There are several questions here so I will try and answer the initial one of
"For example the test that deletes a resource will fail if run the second time. "

Ideally a good practice is for testing to have its own database. The database may need to be seeded with reference data but for the core transactional data, such as that involved in tests, the relevant database tables should be empty before and after each test. You should create factories that can do this data creation.

Your question ask about what to do with data once a test is completed. As you indicate, if you don't have the database empty, you can't make that happen through the api unless it exposes delete functionality.
If you can't get database access then you will need to write tests that compare the database state between tests and write tests that create a new record before deleting it through the api, etc. Basically you would end up spending a lot of time on this issue and it would get worse and more complicated over time, greatly hampering your ability to do application development. This need to be communicated very clearly to management. Do that know or bear the consequences as development slows down and you are blamed for that. For this case a delete route that correctly encapsulates issues such as referential integrity would certainly be better than writing your own deletes.

  • I personally would prefer to have something similar to the DELETE /changes/to/test/user/ route that will reset the db and other services for the test user, as our stack is quite complicated (multiple microservices, db, external api's, external file storage, and all configured differently for each env: local, dev/acceptance, stage), and when developing tests, especially the people that work with the front-end don't have access to the db only the API, and they shouldn't know where and how things are stored in order to make the tests. – Stefan Rogin Dec 23 '16 at 12:54
  • Also our CI environment is run on a separate server from our stage/acceptance environment and exposing direct DB access is not easy and will require maintenance for every change. – Stefan Rogin Dec 23 '16 at 12:54
  • Yes I think the individual delete route is a good idea. That way, even without a db truncate strategy, you can at least pursue a db transaction based strategy for entries that are rolled back or negated through a delete process. – Michael Durrant Dec 23 '16 at 13:30
  • Also I'm not too focused on breaks the responsibility principles because rules are great and you can break them when you know why you are doing it. – Michael Durrant Dec 23 '16 at 13:32
  • was thinking more of a reset route only for the test user, so that the business rules to not be broken, that will remove all the data for that user and reseed with the default test data required. As the API has access to all the required resources to do that command, and the tester doesn't. – Stefan Rogin Dec 23 '16 at 14:24
0

You could use an in-memory or flat-file database like SQLite. It should be easy to use in your test environment.

If your app has a dependency on a particular database, that could get trickier. In that case, you have no choice but to tell your manager you can't run integration tests without, well, integration. Your CI server should be fully capable of running an instance of your production database systems for you to test against- if not, this is a serious problem.

Best choice for tests is to set up sample data and then run the test inside a transaction that is always rolled back. Then each test should see an unmodified copy of the database.

Edit:

This holds true for all your deps. You need to have deps in development and in test. This is why people try to minimize their deps. Either remove the dependency or have it present in dev and test.

  • and how would I run the tests locally ? – Stefan Rogin Dec 23 '16 at 14:34
  • also check the update in the question ? – Stefan Rogin Dec 23 '16 at 14:34
  • If your app has those dependencies in production, you need them in testing. That's one of the bigger arguments against taking such dependencies, but since you have them, you need to either remove them or live with them. The best you could do is fork out using them into separate projects/repos and test them individually. – DeadMG Dec 24 '16 at 15:03
  • The projects are in their separate repos, but they do have events that send commands from one micro-service to another, and that's why I emphasize on integration tests, as the rest we have covered. – Stefan Rogin Dec 26 '16 at 20:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.