Many small requests vs. few large requests (API Design)

Question

I'm currently working on a project with an organization as follows:

• Client - Gets data from the main server via REST api.
• Server - Requests data from various other servers via third-party APIs
• Third-party APIs - Services out of my control that provide data to the server (Reddit, Hackernews, Quora, etc.)

For the sake of argument, let's say the client first needs a list of items from each of the third-party APIs. From this list, an item will be chosen at which point the client needs to see the full content of the item as well as responses (i.e. comments) to the item. I'm trying to decide between three options:

A la Carte

In this approach, I would have 3 separate endpoints on my server: one to get the list of items, one to get the main content for an item, and one to get the responses of the item.

• Pros: I never make more requests than I need to, requests should be small so generally they should be faster.
• Cons: I have to make a lot of requests. After choosing an item from the list, the user may have to wait before seeing the main content and then wait even longer to see the responses

Server-side cache

In this request, I would make a single call to my server to "fetch" all the data for all sources. The data would then be cached on the server. The client would then have the same REST endpoints as before, except there wouldn't be much waiting between calls since my server already has the data and just has to feed it to the client.

• Pros: Still easy to implement on the client side, but without the latency issues
• Cons: A bit more involved server side, and the first call could take really really long.

Client-side cache

This scenario is similar to the previous one except the client only ever makes one request to the server: give me all of the data. From here it's the client's responsibility to save the data and use it appropriately.

• Pros: Easy server implementation, very speedy after the first call
• Cons: First call will be very slow, more complicated client-side implementation

I'm not sure which is the best approach, or if maybe I'm missing the obvious solution. Any advice would be greatly appreciated!

Answer 1

One thing to keep in mind is the expected network latency (i.e. ping time) between your clients and your server. In a high-latency situation with otherwise good bandwidth, many small requests will perform significantly worse than one large one.

I've recently been collaborating on a multi-team database-backed web application project in which one of the teams is in India (the rest are in the US). We have a single database instance hosted in our US office to which developers connect our local webserver instances. My desk is maybe fifty feet and two LAN hops away from the database instance, and performance is fine.

When we first got things started with the developers in India, they were experiencing enormous wait times starting up the application and navigating page-to-page. We're talking ten minute wait times here. It turns out that this was because the ~200ms ping time from their desks to our dev database server was getting multiplied by many, many brief queries to the database. My local 0.5ms ping was so trivial that the chattiness between webserver and database server never mattered. This was the first time that we had geographic separation between webserver and database server.

The solution in our case was to clone the database server and stand the copy up in India, but the point here is to keep in mind that if your client and server are far apart, network latency will be multiplied by the number of communications across the wire. Bandwidth once the connection is established is typically much less of a concern.

Answer 2

These three options are not mutually exclusive, you can use an combination of both client-side and server-side caches. However some of data, like comments, may become stale, if kept in cache for too long. Considering you can't check whether that is the case, you should probably refrain from storing it at all. On the other hand content usually doesn't change drastically, so there would be no harm in caching it server-side and then prefetching some of it on the client-side to lower the latency.

Answer 3

I would (almost) discount option 3. Choosing between 1 and 2 depends on two things:

• (A) how big the result of a single, total fetch is
• (B) how much of the detail of the result the client/user will typically use in that session.

It's easy to make a decision if A and B are extremes:

• If A is large and B is small, definitely go for option 1 (A la Carte).
• If A is small and B is large, go for 2 (Server-side cache) or even 3 (Client-side cache).

For any other A/B (large/small) variations, you will have to apply discretion. I often provide both coarse and fine endpoints to cater for different use cases from different clients.

Answer 4

Based on just the info you gave, option 1, because

• with a single client request you'd be mixing apples and oranges and the fruit basket might be very large.

• caching is a tradeoff where you gain performance but potentially lose consistency (stale data). If you don't have any identified performance issues, synchronization problems are usually not worth risking.

Answer 5

I have always found a few large requests to be better performing and more scalable. But there are tradeoffs in all approaches, so it depends on the needs of the server and client. You may wish to use another option, which is to have the client specify a whole range or set of data to retrieve -- not necessarily all the data, but some range, which is tuned over time to match the bandwidth available.

Answer 6

As always in programming, it depends.

So, the real question is: what should you consider when deciding for A/B/C or a combination of the three?

I would say that the real discriminating factors are the implementation details of the 3rd party APIs you are consuming. As an example, you should consider: Are they fast or slow? Do data changes frequently and unexpectedly? Are they "chatty" or Rest?

In case of fast, easy to call services, with data changing so frequently that your server side cache is going to create stale-cache problems, by all means, go for option 1: more requests, no cache, only when needed.

If your external data is going to change in a predictable way, or you are limited with the usage, or simply you can get a better user experience caching data on your server, go with 2. But keep in mind that cache is not free: it has costs in terms of debugging and sometimes users complaining they don't see updates.

Option 3, I would consider only if data is not much, but in that case even options 1 or 2 can work, and you keep more logic on the server, so I would stick for 1 or 2.

Just my 2c.