3

Research leads me to believe the most common architecture for a multi-instance (horizontally-scaling) message relay service (the typical example being an instant messenger/chat service) is to distribute client connections among the instances via some load balancing strategy. The instances hosting the clients wishing to converse then communicate internally (discovering each other (e.g. via common database or peer-sharing across a mesh) and communicating directly, or messaging through a common broker). Rough diagram of typical architecture:

typical architecture diagram

Why incur the extra, ongoing load of the inter-instance communication for the duration of the session when instead the load-balancer/router could initially route both clients to the same instance? The router would make the same database lookups that an instances would have in the former scheme, and you would save a connection between instances (or in the case of a shared broker, two connections and the consideration of scaling the message queue as well), while decreasing end-to-end transmission delay.

proposed architecture diagram

Is this architecture common, and I just haven't read about it, or does it have a downside?

Improve this question
3
  • 1
    How does the load balancer know which clients are going to communicate with each other?
    – user253751
    Sep 3, 2020 at 20:45
  • 2
    It sounds like you are describing something akin to session affinity in your load balancer. That said, when you put more logic in your load-balancer to route traffic (i.e. session affinity) you'll find that there is a limit to how well you can scale. There are a number of other places where you can potentially scale as well, such as a Redis cache to hold the messages.
    – Berin Loritsch
    Apr 27, 2022 at 1:23
  • your second schema seems quite broken: why would you need a router if everything goes to the same instance? why does the balancer need to interact with the database? what's the point of an instance that doesn't communicate with the DB? If what you call router makes all the DB interaction than router and instance are the same thing, which is just a plain single-instance architecture without any horizontal scaling at all.
    – Carlo Moretti
    Aug 25, 2022 at 13:02

1 Answer 1

Reset to default
0

To be honest, I don't really like the way this question is possed but I'll accept all the facts axiomatically (without challenging them!).

Your second design should require a more advanced handling protocol, should result in not so optimal resource allocation (see serving queues mathematics) unless somehow handled and maybe other shortcomings, while its benefit should be fewer connections thus higher performance.

There could be scenarios that the benefit of the fewer connections is negated by the non optimal resource allocation, and maybe also negates the whole horizontal scaling solution(someone will pay for this and needs to be convinced that this solution is what they need).

Another scenario could be geographical distribution of brokers and one instance initiated in a bad positioned server will cascade to the rest of the session.

So to conclude, while not knowing all the determinants of the first design, it seems a bit more generic, basic, and less complex .An alternative design could be considered within specific scenarios of course.

Improve this answer
5
  • Evidently I've not provided enough information, or made too many generalizations to allow you to provide useful analysis. Would you be so kind as to comment in what aspects the question could be refined? Good note on geographical considerations!
    – Chase Patterson
    Dec 8, 2019 at 17:27
  • 1
    Hey pal. You begin by setting up a context, which is somehow ambiguous "Research leads me to believe". There could be many different ways a solution/design is defined and b) there could be many different layers this can occur upon. F.e. Application and network layer. I think your question is a mix of these. Then you point out a perceived shortcoming (extra connection) and propose a solution, which is obviously less cost effective in the regard that there should be extra logic to support this solution, soo one trade off should be apparent enough in your question.
    – Cap Barracudas
    Dec 9, 2019 at 10:47
  • I could continue, but this is in general what I was referring to as "I do not really like how this question is posed". But I do not mean to discourage you, your questioning ability seems already good enough, just it might need tiny enhancements.
    – Cap Barracudas
    Dec 9, 2019 at 10:47
  • Plus, I think the solution you propose is called "sticky sessions" but everything is a solution to a specific problem, which in here is not so obvious. Soo, I could say your question is a bit vague too.
    – Cap Barracudas
    Dec 9, 2019 at 10:50
  • I see I was being a bit too abstract, and should outline more of the specific technologies and process flow that I envisioned in this scenario. Also, some of the related solutions I've investigated and how I perceive they fit/don't fit (e.g. session affinity)
    – Chase Patterson
    Dec 9, 2019 at 17:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.