Advantages of using a message broker for scaling websocket client-to-client communications

Question

I'm designing a system where pairs of clients need to exchange messages proxied by a backend service. My initial plan was to use websockets and have the clients connect to a single websocket server, but this doesn't scale obviously. One popular design for scaling websocket servers is to use multiple websocket servers, and broker messages on the back end with some sort of pub/sub message broker (redis, or similar).

Given that my requirement is only for pairs of clients to communicate (no chat-like behavior), and I only need at-most-once delivery, is there any advantage to using a message broker over just having the hosts communicate directly? Or phrased differently, is there any reason not to just have the servers communicate directly?

i.e., in this context, are there any advantages to this:

over this:

To clarify the question a bit more, my concern is that a message broker adds an extra bit of infrastructure without the expected gains in scalability in this scenario. In most uses where a message broker is recommended, there is 1-to-many communication (chat apps, multiplayer gaming etc), where a message broker could add pub/sub capabilities, or message buffering/durability. In this case, I have only pairs of clients, and fire-and-forget messaging is adequate. It would also be trivial to have the servers hosting each side of the conversations exchange messages directly. So I'm wondering if there are other advantages to message brokers that I am missing in this context.

For reference, I'm anticipating several thousand concurrent sessions each consisting of a pair of clients.

Answer 1

Aside from pub-sub (which isn't relevant here,) the main problem that these systems solve is the scenario where one peer tries to send a message to the other and the target isn't able to receive it or confirm the message etc.

You say that you need 'at most once' delivery. That implies that it's OK if a message never reaches it's destination. If that's really true, then you might be able to go without this component here. If you can't send the message, it's a failure and it will never be delivered. There are a lot of ways that things can go wrong but if it really doesn't matter if the message is never delivered, then you might be OK.

Of course, there's a difference between 'a few messages never reach their recipient' and 'it is a regular occurrence that messages never reach their recipient'. A dedicated component for this can help improve reliability even if you aren't strictly concerned about it.

Answer 2

Whether you like it or not, this program is a Message Broker.

As you've already noted, you could implement that functionality directly, or pass that functionality off to a third-party message broker.

If you choose to implement you're own solution you get all the advantages of writing your own code. You have complete control over what it does and how. You also have all of its disadvantages, you will need to become a message broker expert and become very interested in a number of resource, network, configuration, synchronisation, etc... issues.

Alternately you can hand the nitty gritty details off to a third-party solution. You'll still need to learn about its API, and configuration so its not free, and you'll need to be aware of its deployment requirements, resources, etc... On the otherhand you no longer have to worry about how it achieves it, and can benefit in the future from features, and optimisations that the third party implement. The downside being that the third-party dictates what is improved, how, and when.

Neither solution is clearly better. Each will be superior in a given context and not in others. If you are a small shop, the messages can tolerate delay, or there happens to be a third-party solution that fits then I would prefer the third-party solution. If you have a larger shop, messages cannot tolerate delay, and no third-party solution fits - or you will only ever deploy 2 servers - then implement your own.

Either way you choose to solve the problem, my first step would be to invert this problem and make the delivery mechanism a plugin to the server. Now I can atleast isolate my tests from the chosen implementation, and also slot in different solutions to see what works better as the systems needs change and grow.