How to Properly Design an API Prefetch System

Question

I'm creating an API for a web service. There are a couple of endpoints in this API that require some time (lets say +5min) to compute, so I'm creating a prefetch system that will call in the background this endpoints every certain period of time and cache the results, when users call this endpoints, they will get the cached response without having to wait a lot of time. The functions that all this endpoints call will do three things:

1. Make some external API calls to fetch raw data.
2. Process the raw data (this is what takes a lot of time)
3. Store the processed data in DB and return it

I'm having lots of issues with this endpoints, because every time this prefetch system goes down or has any issue, they get exposed to all users and if a user make a request and there is no cached response, then the endpoint will start the computation that takes a lot of time. I'm not sure how should I handle this, I could add a parameter prefetch to only query and process the data if it is called from the prefetch system, otherwise just query the DB for the stored processed data, if there is no data in DB it will return an empty response, but I think this is way better than what happens now.

Here is an example of the function for a given endpoint in Python:

@cache
def calculate_weather():
    # Call some APIs and do some operations
    # Store the processed data in DB
    # return the processed data

This is what I have in mind doing:

@cache
def calculate_weather(prefetch):
    if prefetch:
        # Call some APIs and do some operations
        # Store the processed data in DB
    else:
        # Get the processed data from DB
    # return the processed data

In this second example, the function that the API endpoint calls would have a boolean parameter prefetch, that will only be equal to True when the function is called form the prefetch system. I think this would solve in the sort term this issue, but maybe I can do something else, any suggestions?

Answer 1

Well

1. I think you should have two different endpoints. An endpoint for your prefetch system, and another endpoint that the user would call. Clearly these are treated differently so they deserve their own methods.

2. There's probably a better way to handle this extra long response time issue. It seems like in your usecase, customers are mostly hitting the cache... why not precompute the raw data and persist it in a DB. It depends on the data, but maybe you can precompute it asynchronously when their account is created for example.