Scaling tcp/ip socket server applications

Question

I have been trying to understand how one would scale an application utilizing tcp/ip sockets for client server communication. For example in the simple diagram below:

        +------------+
        | Server     |
        +------------+
           |      |
           |      |
           |tcp/ip|
           |socket|
          connections 
           |      |
           |      |
+------------+  +------------+
| Client_001 |  | Client_002 |
+------------+  +------------+

Two clients are connecting to one server. An action can be performed, and the result can be sent to all clients connected to the server. This makes perfect sense. However if one were to introduce additional application servers behind a load balancer like such:

+------------+  +------------+
| server_001 |  | server_002 |
+------------+  +------------+
          |       |
          |       |
        +------------+
        | HAProxy    |
        +------------+
           |      |
           |      |
           |tcp/ip|
           |socket|
          connections 
           |      |
           |      |
+------------+  +------------+
| Client_001 |  | Client_002 |
+------------+  +------------+

The clients would be making connections to the load balancer. The load balancer would then be delegating the connections to one of the two application servers.

Lets pretend that client_001 executes the program and the load balancer connects them to server_001. Then client_002 executes the program and the load balancer connects them to server_002.

server_001 performs an action on behalf of client_001 and sends a response to all connected clients. However, since client_002 is connected to server_002, it does not receive the response.

Which leads me to my question: How can you send a response to all connected clients independent of which server they were connected to via the load balancer?

Answer 1

I would recommend looking at distributed pub/sub frameworks. Even if you decide to implement it yourself with TCP/IP instead of using the framework, their architecture should still provide inspiration. ZeroMQ handles this use case, and so does Redis.

You can read about how ZeroMQ handles pub/sub here.

Redis does distribution by sharding the dataset into chunks. Each Redis server is responsible for a shard. You would try to design your shards in such a way that clients will be fairly evenly distributed among shards.

You can have server_001 connect to server_002 as a client and when it receives a message, it can relay the message to all clients connected to it.