0

Scenario:-

  • The load balancer distributes traffic to backends with a simple round-robin mechanism.
  • With default config, each backend is assigned weight "1" so all backends are given equal chance of receiving requests.
  • If some backend is given weight 2, then that backend is given twice the number of connections.

Hence, changing weights is a way to make sure all backends have the same load.

Problem Statement:-

Based on the load feedback from backend, define the weight of each backend such that all backends have a very similar distribution of requests

For eg, consider we've 3 backends A, B, C.

  • If the backends have a similar load, then weights stay 1, 1, 1.
  • If A & B have load "L", but C has load "0.8L", weights should increase for C so that it can ultimately catch up and reach load L in future.

Actual example:

In my case, this is a TCP load balancer (L4) acting as the first point behind multiple L7 load balancers.

TCP Load Balancer -> LVS (ipvs+linux)
Backend -> Nginx

Is there a proposed well-known algorithm on how should the weight be chosen to ensure that all backends will ultimately achieve similar load in future?

  • 1
    It seems that by load you mean utilization but you are using the terms load and weight too loosely. Please properly define them first. – Martin Maat May 9 at 6:34
  • Rather then trying to manage this actively (which seems hard to do right in real time) I would just queue all requests and have systems pull from the queue when they are idle. – Martin Maat May 9 at 7:23
  • @MartinMaat Thanks for replying. Something like a queue is not possible in my case. In my case, this is a TCP load balancer and there's no queue and it'll be slow if it was ever implemented like that. What you're essentially saying is using an algorithm called JSQ (Join the shortest queue) or LOR (Least Outstanding Requests). I've updated the question with the exact implementation that I am working on. – Abhijeet Rastogi May 9 at 12:17
  • You're looking for the research area "Control Theory". Core concern will be how fast your system can measure the load and adapt the weights, if it were infinitely fast it could just set all nodes to weight=1 and all overloaded nodes to weight=0 until they are no longer overloaded. – amon May 9 at 15:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.