I have a question regarding load balancing of a service. My load balancing requirement is as follows:

  1. I will have multiple application servers for a non-http service. Application servers will be behind a load balancer(layer 4).
  2. A client will initiate a connection to the load balancer and will send multiple requests over this same connection.
  3. Load balancer need to distribute the requests from this single client connection to the multiple application servers in the backend.

Will round robin load balancing strategy work in this case? Does round robin load balancing strategy distribute only connection requests from client or it distributes the work load requests also?

Please suggest any load balancers which support such load balancing requirement?

  • 2
    this is more of a networking question. But my understanding is that a Level 4 LB won't be able to separate out the requests from a single connection. But you dont really provide enough information to be sure
    – Ewan
    Commented Jun 23 at 14:16
  • How many clients will work simultaneously?
    – Basilevs
    Commented Jun 23 at 15:37
  • suppose i have only one client
    – dsingh
    Commented Jun 23 at 16:02

2 Answers 2


Level 4 load balancers won't help you here. These can only distribute load based on transport information, not based on application protocol. In particular it is not aware that your single connection is composed of multiple requests and can be split. Especially if that is a custom, non-HTTP protocol. It won't do that, it will simply forward that entire connection to one of the target servers.

That being said, the above is not always true. It is sometimes possible. For example if your custom protocol lives on top of QUIC (which is considered a transport protocol, even though it is a protocol on top of UDP), then there is enough information to distribute requests (assuming one request per one QUIC stream). One of the reasons why QUIC is so cool. And it seems that at least HAProxy does support such QUIC load balancing, although recompilation might be required: https://www.haproxy.com/blog/how-to-enable-quic-load-balancing-on-haproxy

But generally you need an application load balancer. And since you don't use HTTP, likely a custom one.

Or you don't. Often transport load balancing is enough even in such case. As long as you have big enough traffic, i.e. enough parallel clients. In such situation it likely won't matter much. Except that you say "single client", which is a slightly weird case.

As for strategies roundrobin is very simple and in most cases good enough.

  • Oops, sorry abot that
    – Basilevs
    Commented Jul 6 at 12:20
  • in my case application protocol used between client and server is gRPC, and i learned recently that gRPC provides api to load balance request originating from a single client connection. I am going to use gRPC api for my load balancing requirement. Thanks for the answers and insights.
    – dsingh
    Commented Jul 11 at 16:18
  • @dsingh dude, gRPC lives on top of HTTP.... I don't see anything hard in load balancing that.
    – freakish
    Commented 2 days ago

Roundrobin will happen by default in HAProxy unless you have sticky sessions enabled. Roundrobin will load balance your single client to multiple backends. I imagine it is the same with NGINX, but my experience is mostly with HAProxy. Yes roundrobin is what you want.

I missed the single connection requirement so the answer above is just wrong since HAProxy will only do either TCP load balancing or HTTP load balancing.

Interesting problem and I like the other answer, but I am not able to comment on it.

To add to their answer, custom work on the server sounds like what you need (without knowing more of the entire goal).

I would probably use a quick API like fastapi and just accept job details in a POST or if the data is too large either just write a socket server in python or use sshkeys and scp to upload the job and write a python watcher of the target directory.

You could then write logic around distributing the workload to other servers. Round robin or even just random for the single connection would probably work. But we need more details about your problem and goals.

  • 1
    As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.
    – Community Bot
    Commented Jul 6 at 6:27
  • It is clear enough, even though the argument is weak.
    – Basilevs
    Commented Jul 6 at 8:14
  • Argument? This isn't a philosophical discussion lol. It doesn't have to be the best solution. Just do your job. Get stuff done. I need something that will load balance per connection. Oh HAProxy works. Use that. Don't waste more time looking for the best possible solution. Optimize once you've gotten the job done and the cost-to-benefit warrants it. That's philosophical. I will make arguments for that. I will not make arguments for a simple solution to a simple problem. Commented Jul 6 at 14:15
  • Oh dear... except that I fear my answer is just actually wrong. Commented Jul 6 at 14:21
  • 1
    HAProxy does not work for splitting multiple requests on a single non-HTTP connection. It can do connection based load balancing, or HTTP request load balancing. Neither is what the op asked for. Commented Jul 6 at 15:09

Not the answer you're looking for? Browse other questions tagged or ask your own question.