0

I'm creating an API for a web service. There are a couple of endpoints in this API that require some time (lets say +5min) to compute, so I'm creating a prefetch system that will call in the background this endpoints every certain period of time and cache the results, when users call this endpoints, they will get the cached response without having to wait a lot of time. The functions that all this endpoints call will do three things:

  1. Make some external API calls to fetch raw data.
  2. Process the raw data (this is what takes a lot of time)
  3. Store the processed data in DB and return it

I'm having lots of issues with this endpoints, because every time this prefetch system goes down or has any issue, they get exposed to all users and if a user make a request and there is no cached response, then the endpoint will start the computation that takes a lot of time. I'm not sure how should I handle this, I could add a parameter prefetch to only query and process the data if it is called from the prefetch system, otherwise just query the DB for the stored processed data, if there is no data in DB it will return an empty response, but I think this is way better than what happens now.

Here is an example of the function for a given endpoint in Python:

@cache
def calculate_weather():
    # Call some APIs and do some operations
    # Store the processed data in DB
    # return the processed data

This is what I have in mind doing:

@cache
def calculate_weather(prefetch):
    if prefetch:
        # Call some APIs and do some operations
        # Store the processed data in DB
    else:
        # Get the processed data from DB
    # return the processed data

In this second example, the function that the API endpoint calls would have a boolean parameter prefetch, that will only be equal to True when the function is called form the prefetch system. I think this would solve in the sort term this issue, but maybe I can do something else, any suggestions?

2
  • 1
    First of all, why not trying to fix the 5+ minute computation is it's clearly the bottleneck and not fit for fast request/response cycles. What does prefetch do? Can you, for example, pre-compute the results somewhere nightly and swap them every day?
    – Bart
    Dec 3 '21 at 19:24
  • @Bart thanks for your comment. I've already tried to lower the computation time as much as I can, but it is still a lot of time. Prefetch is a background task that simply calls the endpoints, the computation is done and saved in the cache, so when users make requests they get the cached response. I will consider moving this to a different machine and swap the data every day as you say, I haven't thought about that.
    – Stack
    Dec 3 '21 at 21:18
1

Well

  1. I think you should have two different endpoints. An endpoint for your prefetch system, and another endpoint that the user would call. Clearly these are treated differently so they deserve their own methods.

  2. There's probably a better way to handle this extra long response time issue. It seems like in your usecase, customers are mostly hitting the cache... why not precompute the raw data and persist it in a DB. It depends on the data, but maybe you can precompute it asynchronously when their account is created for example.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.