In one of my interviews I was asked a vague question, I'm still not sure of the answer. We have a client and a server they are placed apart. The network that connects them has high latency, which is dominating the performance. How can we improve the performance. Note that we can't change the network's topology. I was thinking of caching, breaking the request to multiple smaller requests and opening multiple connections with the server. Any ideas?

Please note that the question description is vague and I wasn't supplied with more information about the situation.

Clarified question: How should the client-server communication be designed in order to get the best performance on a network that has big latency?

  • 5
    "Please note that the question description is vague and I wasn't supplied with more information about the situation." Then consider the possibility that you were expected to ask for more information.
    – user16764
    Feb 6 '13 at 0:44
  • @user16764 I did, but I was told that it was an "open question"
    – Mike G
    Feb 6 '13 at 0:44
  • 3
    Doesn't seem like a vague question. They're asking if you know what latency is and how to minimize its effect. A good answer will start: High latency means that _________. To prevent that from being a problem, we could __________...
    – Caleb
    Feb 6 '13 at 2:27

If high latency is screwing up performance, I'd do the exact opposite of what you're suggesting: find ways to combine multiple requests into a single request.

Let's say latency is 1 second, and you need to process 100 items, and the actual processing time is 0.01 seconds per item.

100 requests
Processing time = 0.01 * 100 =   1 second
Latency         = 1 * 100    = 100 seconds
Total time                   = 101 seconds

But if you can find a way to send two items in a single request:

50 requests
Processing time = 0.01 * 100 =  1 second
Latency         = 1 * 50     = 50 seconds
Total time                   = 51 seconds

Congratulations! You just cut the runtime for this batch in half. Or, if you can find a way to send all 100 items in a single request:

1 request
Processing time = 0.01 * 100 = 1 second
Latency         = 1 * 1      = 1 second
Total time                   = 2 seconds

As a few commenters have noted, this only makes sense if the problem actually is latency, and not other network-related issues. But since that's what you were asked about, this is the right way to handle it.

  • You should probably say something about transmission time (as opposed to latency.) You've doubled transmission time. In high latency situations, this is still the right thing to do, but it's a tradeoff worth mentioning that probably won't apply for batches of two, but if you batch all 100 requests in one request, it might. Feb 6 '13 at 1:00
  • High latency does not mean that you cannot initiate a new request before the previous request is serviced. You can only get away with that kind of synchronous behavior under low latency. Although you are right in that it's probably pointless to break up a large request into smaller ones because of latency. Doing that because of a low bandwidth, or low link reliability would make sense.
    – Kaz
    Feb 6 '13 at 1:01
  • 1
    Take that to its logical extreme, add in some safeguards against packet loss or overloading the destination server, and you'll get something very similar to the sliding window protocol :)
    – Tacroy
    Feb 6 '13 at 1:08
  • @Kaz It's not just pointless...breaking up a few large requests into more smaller ones will actively slow things down. Latency is a cost that is paid per request. (Though obviously making parallel costs means you pay the price in parallel.) Feb 6 '13 at 1:08
  • 2
    @StevenBurnap - if you've been told that latency is dominating the performance I think it's reasonable to assume that transmission time is not such a major factor. Feb 6 '13 at 1:11

Either load more data up front so you dont need to go back again until there's an update.

Use a local data store and sync when appropriate if it's mostly local data

Make lots of targeting small requests, small data going across the wire

and compression.

I probably would've thought of a funny answer... personality goes a long way.

"I'll just use my iPhone, i've got unlimited data and these 4g speeds are GREAT!"

A few ideas that come to mind:

  1. Compression - is the data being sent back and forth compressed to make it as small as possible?

  2. Caching - could there be caches set up on the client to try to make things appear faster? Could the system be set up so the client can operate disconnected from the server for a time?

  3. Network connectivity - While you can't change the topology, it may be worth identifying if there is something else here that could be changed?

While it is an open question, there is something to be said for what is in this client/server system. Are there database calls? Is there synchronization to be set up? Are there alternative architectures that may work better?

The question then becomes how much can compression affect things. If the compression can mean that 100th or 1000th as much data has to be sent on the wire, I'd think that could be a factor as sending so much less may change things. The issue with the question is that there are many unknowns.

  • 2
    Compression doesn't help much if the dominating factor is latency. Sure, there'll be less time between when you transmit the first bit and when you transmit the last bit, but if it takes ten minutes for the whole packet to round trip that becomes a really insignificant factor.
    – Tacroy
    Feb 6 '13 at 1:04

Latency and bandwidth are two different things; high latency means there's a time overhead in sending messages back and forth, while low bandwidth means that the pipeline has a limit restricting synchronous data transfer. Both of these can mean it will take a while to get data, but for different reasons.

The solution for high latency if bandwidth is not an issue is to use larger requests for data; as much as you know you'll need, to minimize the additive impact that the overhead delay will have. The solution for low bandwidth if latency is not an issue is to use smaller requests for data, as little as you can get away with at one time, to reduce the delay that completely retrieving a large dataset can add. In addition, for both cases, you should work as asynchronously as possible, so that you can give the user feedback while the data pull is occurring. Caching is an option if the data is relatively static and is not sensitive (i.e. anyone else who could get into the system could see exactly the same data; stock ticker = cache, bank account balances = don't cache).


On the network with big latency, at first place you have to organize you protocols asynchronously. That means that you have to try to work without waiting for the response of the server.

You have to send the requests and to not wait for response. Then you have to organize separate thread or something that to process the responses when they come.

This way you will use the whole speed of the network and will get the respective responses only delayer in time with double the latency time.

Of course, it is possible only if the requests does not depends on each other and this is your task to design such workflow of the application.

Not the answer you're looking for? Browse other questions tagged or ask your own question.