Persistence layer in the automated testing of an API

Question

The automated tests of an API should be stable and simple.

When writing automated tests for an API, we often have to check that the data created with a program implementing this API is persistent. If the persistence layer is checked directly (for example: a database), then the test is tied to the usage of this persistence layer, and could break often.

We'd rather use the API to check that the data was persisted, so that the implementation could change without breaking the tests.

The example

Let's say we are testing an API to add users, "myfakeapi.com". Given this JSON file "user.json":

{
  "name": "Harry"
}

... And this testing script:

#!/bin/bash

# Deleting all the users
curl -X DELETE https://myfakeapi.com/users

# Adding the new user, returns {"id": 1, "name": "Harry"}
curl -X POST -H "Content-type: application/json" --data-binary "@user.json" https://myfakeapi.com/users

# Checking that the user was saved, returns {"id": 1, "name": "Harry"}
curl -X GET -H "Content-type: application/json" https://myfakeapi.com/users/1

We could say that the test is successful. In addition to that, it wouldn't break if the implementation changed.

The issue is that nothing proves that myfakeapi.com stores the user in a database. In fact, it could pass the tests by storing the user in memory. In the end, was checking the persistent layer directly the right solution?

I don't think so (EDIT: I changed my mind after).

The question

From the point of view of the API's user, persistence only means that if there is a reboot of the program, the data isn't lost. This is exactly what I would like to test.

My question: Is it OK to reboot the program that implements the API during the testing session to check persistence?

Answer 1

As an API user, I only care that writes (POST/PUT/DELETE) have the advertised effects and that reads (GET) give the expected data, where the time between the various requests is not a factor (unless explicitly documented as a timeout).

How the provider of the API makes sure that time is irrelevant is not interesting to me. As far as I am concerned, if they can make sure that the executable never dies, then they can keep all the data in RAM.

As implementer of the API, I may want a more persistent way of storing the data than just RAM.

From a testing perspective, this means that API tests should check that the API works as expected, but they should not routinely do actions that an API user in production can't do, like restart the program.

Tests if the persistence strategy works correctly, I would expect to find in the unittests, component tests and integration tests of the software.