3

I am trying to write a rest GET api to serve query results. The entire flow happening on the server is:

  1. Fetch user data
  2. Call the first query, if results generated: return results, else continue
  3. Call the second query, if results generated: return results, else continue
  4. And and continue so on.

The problem is I want to remember which query generated the results and next time the user makes the request, call and start the chain from there instead of the previous ones. Also since the step 1 always generates the same data for that specific user, I want to skip this too in the next queries. What would be the best practice for this?

The endpoint suggests a certain items to the user like recommendations. To generate these, I fetch the user seen items and execute certain queries e.g first based on type, second based on price and so on. The user data helps in these queries e.g prices of things he saw, type of products and so on. Since the user seen items are sensitive, I don't want to send it back to the client.

I thought of sending some helper data like query number back to the client (mobile app) and also the user data from step 1 back to the user and sending it back to the server from the client while making the subsequent calls but the I don't want to send user sensitive data from the server to the client.

Another way I thought of was using a cache like Redis but the problem is that adding a cache on Heroku will be costly. Any other way?

|improve this question
  • Does that flow happen on the server side (one API call) or the client side (multiple API calls)? What HTTP verb are you using for the call(s)? Is the client running in a browser? Do you own all the clients in addition to the API? – Eric Stein Mar 30 '18 at 15:40
  • @EricStein The flow happens on the server, the HTTP verb is GET, the client is running in a mobile app environment and I don't own all the clients. – mik dass Mar 30 '18 at 15:49
1

Add an optional query parameter to your endpoint which specifies what step of the process they're on. In each response from the API, include a Link header. Have the target URI include the query parameter for the next step. Use the rel name next to indicate that request is the next one in the series. Clients can follow that link to make the next call in the sequence. Clients can leave off the query parameter to start from the beginning, or they can follow the link to get the next step. You don't have to pass any other information down to the client.

As far as the cost of generating the user data, first test to make sure it's a real problem. If so, you can store it in a simple in-memory cache or in a database table.

|improve this answer
  • Thanks for the solution. So fetching from the cache and fetching from the database, do you think both will have almost the same CPU requirements? – mik dass Mar 30 '18 at 17:36
  • @mikdass CPU? I'd guess so. If you see a performance problem, test? – Eric Stein Mar 30 '18 at 18:28
  • Sometimes (often even?) storing the results in some sort of cache is preferred anyway, not just for performance but also to prevent the data from changing between calls. Person A calls and generates a dataset, Person B updates some data, Person A gets page two of the now updated data, so the results are out of sync. If you persist the results of the initial call, the set that was generated is unchanged regardless of what Person B might have done in the interim. When they start back at Step1, generate and persist a new set. Include a set timeout/expiration if you like. – jleach Mar 30 '18 at 18:51
  • @jleach I am already thinking about a expiration strategy and I don't think syncing is a big problem in my case as the user data are independent for each user. Still thanks. – mik dass Mar 30 '18 at 19:27
  • @EricStein I think testing will be the right way to go here. Thanks. – mik dass Mar 30 '18 at 19:29
0

HTTP is inherently a stateless protocol and trying to hack together an endpoint that returns different results depending on the number of times you call it is a really bad idea.

Why? You are creating a system that has unreliable results and the client doesn't have control over them.

The solution
I think you are looking for API pagination. Have the user send the page number they want to retrieve in the request body. Then depending on the page they are requesting, you can run the correct query.

This is an example of how the request with pagination would look like: 

{
    "query": "john",
    "page": 3,
    "limit": 10
}

We are basically telling the server that we want 10 results per page, and that we want to see page 3 of the results. And implementing this on the backend should be fairly easy with the use of your database offset and limit features.

|improve this answer
  • Thanks for the pagination idea, any idea about the user fetch in step 1? Should I send the sensitive data back to the user? – mik dass Mar 30 '18 at 17:28
  • @mikdass Can you give us some context on this user data? What does the endpoint do and why does it send this data, and what does the data even contain? – IgorR Mar 30 '18 at 18:06
  • The endpoint suggests a certain items to the user like recommendations. To generate these, I fetch the user seen items and execute certain queries e.g first based on type, second based on price and so on. The user data helps in these queries e.g prices of things he saw, type of products and so on. Since the user seen items are sensitive, I don't want to send it back to the client. Is my current approach wrong? Is there a better way? – mik dass Mar 30 '18 at 18:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.