Does it make sense for an API with 1 daily job to do to be asynchronous?

Question

This is a conceptual question about whether my specific use-case warrants the use of an asynchronous API.

Language: Python 3.11 Framework: FastAPI (ASGI)

I believe I am confused about what an asynchronous API really means, differentiated from asynchronous processing (multi-threading, event-loops, futures).

I am under the impression that the strength of an asynchronous API is that it can handle multiple requests at once, leading to less request timeouts.

Because we wanted to avoid request timeouts, I was asked to make a certain API asynchronous. This API only had 1 job to do once a day, but the computations took around 1 hour to complete. By the time the computation was done, the service awaiting for our response would've timed out...

The understanding on our part was that the app will:

1. Receive request
2. Immediately respond with 202 Accepted (so that the request is not timedout)
3. Process request after responding
4. Send the result

Is this correct or an absolutely erroneous understanding of what/how asynchronous APIs operate?

In our use case does it even make sense to worry about asynchronicity?

Answer 1

Think of this as two distinct responses. A response to the initial request and then a response to the result. The result response will not be on the initial request.

When ordering fast food, you have an interaction with a cashier and get a ticket with your order. At some point your order completes and you are notified and pick up your order. Two separate events. Also, you don't keep standing in the order line waiting and waiting (and potentially block other people from ordering).

Similar process here, first you are going to submit your "order" / request. Behind the scenes a message gets dropped in a queue (or similar) and then you will return the order ID of the request to caller. Another process will pick it up and process it.

At some point the process will error/succeed. At that point, the client will be notified. Notification could be email, text message, a message using web sockets, etc. One can also provide an API to check on status of past requests as well since you can use the original order Id.

Most likely the original request will be asynchronous, but short lived. Ideally, the submission process should only take at most a few seconds.

Your "API" may have 3 methods:

• Post to Submit a Request
• Get to Check Request Status (By Id)
• Get to retrieve the result of request (By Id)

One can use polling or push method to get the result. A push would be more efficient.

Answer 2

If caller should be able to monitor status of the task and be able to find out whether final result (eg. completed or failed), you can:

1. return response with status code 303 and Location header status code 303 and information about the task (when it was started, hyperlink to url in location header, how far along it is). You can alse return Retry-After header to indicate that user should wait a bit before trying the url.
2. url in location would return 303 response leading back to itself if task is still running or to final result if it is completed.

Answer 3

I am under the impression that the strength of an asynchronous API is that it can handle multiple requests at once, leading to less request timeouts.

Not exactly. A synchronous API can also handle multiple requests. The main reason for implementing an async API is to avoid long response times. Super-long response times are generally problematic and in some environments they are limited. For example, the AWS API Gateway will fail any request that doesn't complete in less than 30 seconds.

You aren't entirely wrong, though. One of the big problems with long response times is that those requests hold server-side resources while they are waiting. If you have a lot of concurrent requests and they are all taking a long time to respond, you will, in general, need more server-side resources to handle that load than if you use an async design.

Another related reason to use async APIs is for uneven loads. If you get unpredictable bursts of activity, it can be very costly to ensure that you will have enough server-side capacity to handle them at any given moment. So even if, under normal load, you respond quickly, you could end up swamped during the heavy period. An async design allows you to buffer requests and (eventually) process them successfully when volume exceeds throughput for some limited period and then returns to normal.

An hour is a really long time for an HTTP call to take. There are many things that can happen in such a timespan. The client could crash or otherwise disconnect, for example. Making this async is a reasonable thing to do. I would add a GET status URL as mentioned in user470365's answer. This is a widely-used approach. You might also want to consider the redirect response as well depending on your situation. It's not strictly necessary, though. You can return a URI in the 202 response content, as an alternative.