I'm trying to use RabbitMQ for two-way one-to-one communication between clients and servers. So, Client 1 should be able to communicate only with Server 1, and vice versa:

enter image description here

The servers stay always connected to RabbitMQ, whereas the clients are started up and shut down on demand. This is because a server should receive all the messages from its client, but if a server sends a message to its client when the client is not present, it's ok if the message is just lost.

Furthermore, there must be a client-server pair for each user of the system.

My first idea to implement this was to create a new RabbitMQ user and vhost for each client-server pair, and two queues in each vhost (one consumed by the server and one consumed by the client).

However, is this scalable? For example, if there are 10k users of the system, there would be 10k RabbitMQ users, 10k vhosts, and 10k permanent queues, connections, channels, and consumers (from the servers).

Can these large numbers of queues and connections be a performance problem for RabbitMQ?

And what would be the best way to implement this behaviour with RabbitMQ? Or is a different technology than AMQP and RabbitMQ better suited for this use case?

  • I don't know RabbitMQ, but ZeroMQ has great documentation on messaging design patterns that may give you some insight on how to build out scalable systems: zguide.zeromq.org/page:all – bitsoflogic Oct 16 '18 at 14:41
  • Why must there be a client-server pair for each user of the system? – Mike Partridge Oct 16 '18 at 15:53
  • @MikePartridge because a server logs in to the user's account of a third-party service and must consistently run to receive updates from this service for this user. So there must be a server for each user of the system. And the clients are Angular applications. – weibeld Oct 16 '18 at 16:10
  • I would expect any messages sent to the client would sit in their queue until read or removed. This would mean when you a client is not connected, you are using more storage for that client than when they are running. This seems strange given your assertion that it's OK for the messages to get lost if a client is not running. I'm familiar with messaging but not RabbitMQ so my premise here could be wrong. If you only care about sending messages to live clients, wouldn't something like websockets be more appropriate? – JimmyJames Nov 15 '18 at 18:08
  • The server-to-client queue is auto-deleted when the client disconnects, so messages sent by the server are just discarded. The main use case is actually clients making RPC calls to the servers. The server-to-client messages are only for live updates in the client interface. Yes, I wonder if WebSockets are more appropriate, or XMPP. But for WebSokets, every server needs a public IP address, right? – weibeld Nov 15 '18 at 18:42

First off, using separate vhosts is a great idea and (if I recall correctly at least) essentially free.

Each queue, exchange and so on is an Erlang process. These add up (slowly). It's been quite a few years since I last worked with RabbitMQ but 100ks of Erlang processes should be do-able.

Eventually that single box in the middle is going to be overloaded. Clustering the RabbitMQ instances might help, but it depends - mirrored queues will increase resilience to node failure while not increasing capacity.

What might help more with capacity is perhaps a consistent hash for the client/server pairs that lands them on a particular RabbitMQ instance. That removes the single box in the middle for all client/server pairs. Even if you only started with a single instance, using a consistent hash to find the RabbitMQ means you can easily add more instances in the middle.

  • Thanks for the answer. Yes, vhosts are free. How would I notice that the RabbitMQ instance gets overloaded? CPU usage or memory? And does it make a difference if the queues are idle most of the time? – weibeld Oct 16 '18 at 16:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.