How to decide if an API should be synchronous or asynchronous?

Question

I am developing a service which will expose some APIs through REST end-point. I have some API which can take a lot of time like say 5 to 10 minutes to complete the requested operation.

I am also writing the client side application which is multi-threaded and it can make multiple requests to server at the same time. If the server API happens to be synchronous, it is still fine for that particular client thread to hang waiting for the response from server. So, the client is not completely stuck, it can continue with other operations.

I understand that if I make it asynchronous, I have to do some more work for API operation status management, providing way for client to get the operation status.

Having said that, I am still not able to decide whether I should make this server API synchronous or asynchronous.

On what basis does one decide to make a API synchronous or asynchronous? Does it depend on how long the API operation will take? Does it depend on whether client can handle blocking call?

Answer 1

HTTP is synchronous in the sense that every request gets a response, but asynchronous in the sense that requests take a long time and that multiple requests might be processed in parallel. Therefore, many HTTP clients and servers are implemented in an asynchronous fashion, but without offering an asynchronous API. Using multiple threads is one way to implement asynchronicity, but there are others like callbacks or event loops.

If you want an asynchronous API with HTTP in the sense that a response is not immediately available, you will have to model the asynchronicity explicitly:

• When a valid request is received, the server immediately responds with a success status, most likely 202 Accepted or 303 See Other.
• The backend then works on the job in the background (maybe as a separate thread, maybe as a separate server).
• The response should describe how the status of the job can be monitored and how the result can be fetched. Later, the client can use this information to retrieve the result.

There are a couple of techniques like long-polling or chunked transfer to push the result as soon as it is available from that resource, but generally requires client cooperation.

The alternative is to use a different protocol than HTTP, in particular a protocol where messages don't need a response. Within the web stack, WebSockets are such a protocol and allow the server to push a response at a later time. However, using such protocols implies a heavier development effort for you because non-HTTP APIs lack tooling and framework support.

Because async APIs are more complicated for everyone involved, they should be avoided. Again, they are not necessary for having async clients or servers. Modelling asynchronicity in your API is appropriate when:

• the request launches a background job that may or may not be processed quickly, or
• creating the response inherently takes some time, or
• you cannot guarantee that a response will be ready before the client times out, probably on the order of 10 to 30 seconds.

For example, the web interface of my WiFi router can reboot the router. This inherently takes some time. They have implemented this in the following asynchronous fashion:

1. I request a reboot.
2. The server responds immediately with a status page.
3. The router initiates the reboot.
4. The status page continuously requests the status in the background. Most of the time, the request will time out because the server has not yet restarted.
5. The router reboots and the server comes back online.
6. On the next status request, the server delivers a status response that the rebooting has completed, and redirects my browser to a new page.

Answer 2

As a rule of thumb, an HTTP request should turn around within milliseconds, not minutes. You should not know and not care about the implementation details of the client and thus how they may be able or unable to handle parallel requests. The biggest problem with several minute long requests is that you actually need to “hold the line” all this time. This may be difficult for several reasons:

• Network timeouts and interruptions may disrupt the request.
• The client must not shut down or cancel the request, which may be difficult for certain clients, e.g. it’s not the default modus operandi for PHP.
• It makes it harder to manage many requests in parallel for the client as it actually needs to allocate and keep resources for every request it makes and cannot let go of the request resource until the server has finished the job.

Once the request has been interrupted for whatever reason, there’s no way for the client to get the result of the operation, and it has to restart the same process, probably again waiting several minutes, leading to a cycle of failure.

Designing the API as asynchronous makes this much more robust. The client can store a job id wherever it wishes and is free to do other things. It can check back in at any time that is convenient for the client. It does not matter whether any single request fails for whatever reason, every operation can be repeated without loss of state.