I've been working on a large distributed system typically with over 10,000 clients living on a single network (LAN) all accessing a central set of API's in the cloud. Not exactly a challenging problem. Where it gets interesting is that each client may be requested to upload a large amount of data anything from a 1 KiB file to a multi GiB file. A single request to upload will happen ~simultaneously across the clients.

Parts of the system that are stochastic:

  • Which clients are alive.
  • How many clients a file is requested from.
  • The size of the file.
  • The number of requests for files.

My question is, are there any good white papers or known algorithms that could be used to coordinate all the clients so that they don't immediately overwhelm the network that they are residing and use up all the bandwidth? The main issue would be the edge router since they have fixed bandwidth. The expected result would be that the clients figure out a way to coordinate with themselves to eventually complete the file request while keeping bandwidth use to a minimum and that they minimize communication with the API during this process.

To date, I am just allowing a fixed number of clients to transmit large amounts of data at once. Not optimal though it works. Standard rate limiting is out as the system cannot just shut down client connections as they are integral to the functionality of the system.

My guess would be that someone working with map-reduce would have run into a similar issue. I have also, look into the possibility of creating a protocol for local and global voting on priority, similar to MongoDB election process.


You have a 100 employee company with a 300x300 backbone to the internet. They have clients installed on all their laptops and servers. Let's imagine each employee has one laptop and there are 10 servers.

If not hard to conceive a situation where there are many files spanning the majority of machines and they have been requested at the same time. This could easily cause a situation where all the bandwidth is being consumed for a brief period of time affecting delivering said files, the is likely to affect the employee's ability to do their jobs.

| improve this question | | | | |
  • What is the actual problem you're trying to solve with regards to having a fixed number of clients? Are you trying to calculate the optimal maximum number for simultaneous client uploads or looking to schedule each client's upload at different times to avoid collisions? – Ben Cottrell Aug 1 '17 at 19:08
  • Why cant you stick them all through a single system and rate limit it? Also consider whether you're over-engineering at the very beginning. You haven't told us what the system does but what is the realistic number of clients performing a simultaneous batch upload? – Ben Aug 1 '17 at 19:09
  • This was explained in the question, it is being rated limited from the perspective that only so many clients can transmit simultaneously. Access to the API cannot be turned off to any of the clients, as in traditional user/developer rate limiting, as the clients are part of the system. The data being uploaded can be anything from a simple 10KiB file to a 4GiB log, the system stochastic. As the question states if 10,000 clients come online at the same time and upload a 4GiB on a single network (LAN) someone is going to notice. – icirellik Aug 1 '17 at 20:24
  • 3
    TCP and therefore HTTP has built-in congestion control: if packets get dropped because a switch in the network or the receiver is overwhelmed, the sender notices the missing ACK and will continue sending at a lower rate. So most likely you don't have to do anything at all. If you're worried about stealing bandwidth from other network users, don't be. It's the job of the network administrator to manage that if necessary, e.g. by implementing QoS and assigning bandwidth quotas. Most enterprise network gear and even many consumer routers support QoS rules. – amon Aug 2 '17 at 10:44
  • Do the clients tell the server how big a file they're uploading? – kdgregory Aug 2 '17 at 15:07

You should implement your service in a way that requests will fail fast if too many resources are being consumed or clients are being served. A circuit breaker is useful here. The server should provide a meaningful response to clients who have attempted so clients know that they try again later (e.g. HTTP 503 response), and clients should use an exponential backoff (with some randomness) to retry requests.

Limiting concurrent connections may be sufficient approach to limit the bandwidth consumption without resorting to actually calculating bandwidth used.

| improve this answer | | | | |
  • Already done, done and done. Was just looking to see if there has been any research or people have had experience in this subject. Having a fixed limit on clients that may to upload and calculating bandwidth used so each client can throttle appropriately and make sure all other clients back off appears to be the best solution. – icirellik Aug 2 '17 at 2:57

This is really a networking problem rather than an application one.

What you can do is identify the types of traffic at a network level. maybe with separate port number? to allow other traffic to be prioritised over it.

| improve this answer | | | | |
  • This is not an untrue statement, sadly I cannot expect companies to change their networks to deprioritize my traffic. – icirellik Aug 3 '17 at 15:44
  • but if its someone elses network you have no idea of whether they need to deprioritise it – Ewan Aug 3 '17 at 16:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.