Research leads me to believe the most common architecture for a multi-instance (horizontally-scaling) message relay service (the typical example being an instant messenger/chat service) is to distribute client connections among the instances via some load balancing strategy. The instances hosting the clients wishing to converse then communicate internally (discovering each other (e.g. via common database or peer-sharing across a mesh) and communicating directly, or messaging through a common broker). Rough diagram of typical architecture:

typical architecture diagram

Why incur the extra, ongoing load of the inter-instance communication for the duration of the session when instead the load-balancer/router could initially route both clients to the same instance? The router would make the same database lookups that an instances would have in the former scheme, and you would save a connection between instances (or in the case of a shared broker, two connections and the consideration of scaling the message queue as well), while decreasing end-to-end transmission delay.

proposed architecture diagram

Is this architecture common, and I just haven't read about it, or does it have a downside?

  • How does the load balancer know which clients are going to communicate with each other?
    – user253751
    Sep 3 '20 at 20:45

To be honest, I don't really like the way this question is possed but I'll accept all the facts axiomatically (without challenging them!).

Your second design should require a more advanced handling protocol, should result in not so optimal resource allocation (see serving queues mathematics) unless somehow handled and maybe other shortcomings, while its benefit should be fewer connections thus higher performance.

There could be scenarios that the benefit of the fewer connections is negated by the non optimal resource allocation, and maybe also negates the whole horizontal scaling solution(someone will pay for this and needs to be convinced that this solution is what they need).

Another scenario could be geographical distribution of brokers and one instance initiated in a bad positioned server will cascade to the rest of the session.

So to conclude, while not knowing all the determinants of the first design, it seems a bit more generic, basic, and less complex .An alternative design could be considered within specific scenarios of course.

  • Evidently I've not provided enough information, or made too many generalizations to allow you to provide useful analysis. Would you be so kind as to comment in what aspects the question could be refined? Good note on geographical considerations! Dec 8 '19 at 17:27
  • 1
    Hey pal. You begin by setting up a context, which is somehow ambiguous "Research leads me to believe". There could be many different ways a solution/design is defined and b) there could be many different layers this can occur upon. F.e. Application and network layer. I think your question is a mix of these. Then you point out a perceived shortcoming (extra connection) and propose a solution, which is obviously less cost effective in the regard that there should be extra logic to support this solution, soo one trade off should be apparent enough in your question. Dec 9 '19 at 10:47
  • I could continue, but this is in general what I was referring to as "I do not really like how this question is posed". But I do not mean to discourage you, your questioning ability seems already good enough, just it might need tiny enhancements. Dec 9 '19 at 10:47
  • Plus, I think the solution you propose is called "sticky sessions" but everything is a solution to a specific problem, which in here is not so obvious. Soo, I could say your question is a bit vague too. Dec 9 '19 at 10:50
  • I see I was being a bit too abstract, and should outline more of the specific technologies and process flow that I envisioned in this scenario. Also, some of the related solutions I've investigated and how I perceive they fit/don't fit (e.g. session affinity) Dec 9 '19 at 17:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.