How to make Request from Client to Server more evenly distributed (over 24 hrs)

Question

I'm trying to write a new API that can support many devices. There will be at least 50k devices in the beginning that will connect to the server using an HTTP API on a daily basis to receive updates.

The number of devices increases with time (with new installations).

The devices need to contact the server once per day to get information.

The time for making this call doesn't matter.

I want to ensure that the request calls from devices are randomly distributed so as to balance the load on the server.

Suppose that in response to the API call, I have information regarding the next request datetime.

What is a good way to find the optimal time for making this call (within next 24 hrs)? Once suggested, all devices should continue calling at that given time every day.

As this will be a global solution, I plan to pick a time much later after regular working hours for the given device. Let's say 10 PM to 5 AM is the suggested time for each region.

Within those 7 hours, how can I evenly distribute the time for any given device?

Some additional info:

• Time will be allocated for the device when it is making a valid request for the first time
• You can manually force the device to make the call using client software, but that does not change the default settings. But it will also be possible to change the time from the server manually (force entry- no logic required).

I have devised a crude way of tracking the log per minute [minuteofday], [frequency], by selecting the one with the lowest frequency. This should log with respect to a constant of 1440 entries. There is always a chance for race situations.

I also need to assign additional resource when needed.

Answer 1

From your question I understand you like to

I want to make sure that the request calls from devices are as randomly distributed so as to make the load on the server as even as possible.

So I would start with giving each device the ability to randomly select the time it should contact the server, and not enforcing this on the server side.

You like your devices to call the API during a 7 hours window starting at 10 PM.

7 hours -> 25200 seconds.

So each device will have a time pick up logic like

if (I already selected my date){
    keep on using it
} else {
   Random number between 0 and 25200;
   10 PM + random number;
   store above time  
}

Answer 2

You are essentially trying to hash a lot of devices into very few buckets (time slots). You are guaranteed to have some collisions.

I mean, even mathematically: with 50k starting devices and even utilizing the entire 24 hours in a day, you'll get at most 1.728 seconds to service each device.

Also, there are a lot of things that can happen, which will cause a device not to honor your request to only call in at the specified time.

So, since conflicts are unavoidable, I'd suggest implementing your API in such a way, that it first checks how busy you are, and either quickly responds asking to come back in 5 minutes (or at the specified time slot, that you expect is going to be less busy) or services the device right away.

You can even simply drop the request without handling it to save time, and have device retry automatically after a random internal timeout elapses.

Although, I'd avoid looking for randomness when it comes to balancing a load like this—random does not necessarily mean equally spread out.

Answer 3

"The time for making this call doesn't matter."

Then why not just let the clients call the API ?

"I want to ensure that the request calls from devices are randomly distributed so as to balance the load on the server."

You cannot randomly distribute calls which has not yet been activated - that would be controlling the future.