I'm designing a system where pairs of clients need to exchange messages proxied by a backend service. My initial plan was to use websockets and have the clients connect to a single websocket server, but this doesn't scale obviously. One popular design for scaling websocket servers is to use multiple websocket servers, and broker messages on the back end with some sort of pub/sub message broker (redis, or similar).

Given that my requirement is only for pairs of clients to communicate (no chat-like behavior), and I only need at-most-once delivery, is there any advantage to using a message broker over just having the hosts communicate directly? Or phrased differently, is there any reason not to just have the servers communicate directly?

i.e., in this context, are there any advantages to this:


over this:


To clarify the question a bit more, my concern is that a message broker adds an extra bit of infrastructure without the expected gains in scalability in this scenario. In most uses where a message broker is recommended, there is 1-to-many communication (chat apps, multiplayer gaming etc), where a message broker could add pub/sub capabilities, or message buffering/durability. In this case, I have only pairs of clients, and fire-and-forget messaging is adequate. It would also be trivial to have the servers hosting each side of the conversations exchange messages directly. So I'm wondering if there are other advantages to message brokers that I am missing in this context.

For reference, I'm anticipating several thousand concurrent sessions each consisting of a pair of clients.

  • Can downvoters please comment? I tried to find the right stackexchange for design questions, and I feel like the question is clearly answerable without speculation or generalizing. – ConditionRacer Mar 24 '19 at 22:42
  • 1
    IIs there any advantage to using a message broker? Yes. You've pointed it out as being scalable to more than one server. Question answered by you. Did you mean to ask a different question? I am not one of the down voters but as an aside, your actual question is nested deep in the second paragraph, as it is the important point it should be readily apparent. Perhaps displayed by itself stand alone, or as the first/last sentence in the post. – Kain0_0 Mar 24 '19 at 23:57
  • The question was the advantages of a message broker over servers communicating directly, it's not clear how a message broker is more scalable in this scenario. Fair point about the question needing to stand out more, I'll make some edits – ConditionRacer Mar 25 '19 at 0:03
  • Ok, instead of two servers, say you have 1000. How would you manage the inter-operability and inter-connectivity among so many? The failover? The routing of the messages? On the other hand, how server A knows about the clients on B to be able to send messages to B's clients? – Laiv Mar 25 '19 at 10:34
  • @Laiv I was planning to have each conversation between pairs identified by an id, and then store something in a database like conversationId/clientA -> hostA, conversationId/clientB -> hostB. Then anytime client A wants to send a message to client B, it sends to the host it's already connected to, that host looks up where the other side of the conversation is connected and forwards the message over. Failover would require one or both clients to reconnect to new hosts who would then update the records for that conversation. – ConditionRacer Mar 26 '19 at 0:13

Aside from pub-sub (which isn't relevant here,) the main problem that these systems solve is the scenario where one peer tries to send a message to the other and the target isn't able to receive it or confirm the message etc.

You say that you need 'at most once' delivery. That implies that it's OK if a message never reaches it's destination. If that's really true, then you might be able to go without this component here. If you can't send the message, it's a failure and it will never be delivered. There are a lot of ways that things can go wrong but if it really doesn't matter if the message is never delivered, then you might be OK.

Of course, there's a difference between 'a few messages never reach their recipient' and 'it is a regular occurrence that messages never reach their recipient'. A dedicated component for this can help improve reliability even if you aren't strictly concerned about it.

| improve this answer | |

Whether you like it or not, this program is a Message Broker.

As you've already noted, you could implement that functionality directly, or pass that functionality off to a third-party message broker.

If you choose to implement you're own solution you get all the advantages of writing your own code. You have complete control over what it does and how. You also have all of its disadvantages, you will need to become a message broker expert and become very interested in a number of resource, network, configuration, synchronisation, etc... issues.

Alternately you can hand the nitty gritty details off to a third-party solution. You'll still need to learn about its API, and configuration so its not free, and you'll need to be aware of its deployment requirements, resources, etc... On the otherhand you no longer have to worry about how it achieves it, and can benefit in the future from features, and optimisations that the third party implement. The downside being that the third-party dictates what is improved, how, and when.

Neither solution is clearly better. Each will be superior in a given context and not in others. If you are a small shop, the messages can tolerate delay, or there happens to be a third-party solution that fits then I would prefer the third-party solution. If you have a larger shop, messages cannot tolerate delay, and no third-party solution fits - or you will only ever deploy 2 servers - then implement your own.

Either way you choose to solve the problem, my first step would be to invert this problem and make the delivery mechanism a plugin to the server. Now I can atleast isolate my tests from the chosen implementation, and also slot in different solutions to see what works better as the systems needs change and grow.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.