Designing a api with chain of queries

Question

I am trying to write a rest GET api to serve query results. The entire flow happening on the server is:

1. Fetch user data
2. Call the first query, if results generated: return results, else continue
3. Call the second query, if results generated: return results, else continue
4. And and continue so on.

The problem is I want to remember which query generated the results and next time the user makes the request, call and start the chain from there instead of the previous ones. Also since the step 1 always generates the same data for that specific user, I want to skip this too in the next queries. What would be the best practice for this?

The endpoint suggests a certain items to the user like recommendations. To generate these, I fetch the user seen items and execute certain queries e.g first based on type, second based on price and so on. The user data helps in these queries e.g prices of things he saw, type of products and so on. Since the user seen items are sensitive, I don't want to send it back to the client.

I thought of sending some helper data like query number back to the client (mobile app) and also the user data from step 1 back to the user and sending it back to the server from the client while making the subsequent calls but the I don't want to send user sensitive data from the server to the client.

Another way I thought of was using a cache like Redis but the problem is that adding a cache on Heroku will be costly. Any other way?

Answer 1

Add an optional query parameter to your endpoint which specifies what step of the process they're on. In each response from the API, include a Link header. Have the target URI include the query parameter for the next step. Use the rel name next to indicate that request is the next one in the series. Clients can follow that link to make the next call in the sequence. Clients can leave off the query parameter to start from the beginning, or they can follow the link to get the next step. You don't have to pass any other information down to the client.

As far as the cost of generating the user data, first test to make sure it's a real problem. If so, you can store it in a simple in-memory cache or in a database table.

Answer 2

HTTP is inherently a stateless protocol and trying to hack together an endpoint that returns different results depending on the number of times you call it is a really bad idea.

Why? You are creating a system that has unreliable results and the client doesn't have control over them.

The solution
I think you are looking for API pagination. Have the user send the page number they want to retrieve in the request body. Then depending on the page they are requesting, you can run the correct query.

This is an example of how the request with pagination would look like:

{
    "query": "john",
    "page": 3,
    "limit": 10
}

We are basically telling the server that we want 10 results per page, and that we want to see page 3 of the results. And implementing this on the backend should be fairly easy with the use of your database offset and limit features.