0

I want to use rabbitmq to share domains events between microservices. Due to the fact that queues are round-robin, I need to define one queue per event source and event consumer.

For example, cars-ms emits the CarRepairedEvent and should be consumed by notifications-ms and trace-ms. I though about using a fanout exchange to emit the event to cars-ms_notifications-ms and cars-ms_trace-ms. I could subscribe from each consumer to every queue related to it but I would prefer the consumer to subscribe to just one queue and receive all events from it:

enter image description here

  • Is this approach reasonable?
  • How can I achieve it?

Thank you all in advance!

1

At work I have worked in a similar requirement that you have proposed here. I'd like to answer your questions and give some more hints here.

First of all for starters please study this link in order for apprehending the Exchange Types available in Rabbit:

https://www.cloudamqp.com/blog/2015-09-03-part4-rabbitmq-for-beginners-exchanges-routing-keys-bindings.html

Second of all, speaking of your proposal design which does NOT correspond to your intentional design with the Fanout Exchange, but The Topic Exchange. Consult this resource that clearly explains your intention in more technical terms and hands-on examples:

https://springbootdev.com/2017/11/12/spring-amqp-rabbitmq-topic-exchange-example-part-1-producer-application/

Now let's discuss some possibilities that you may have. I have drawn a simple illustration below about what you can achieve using Rabbit:

Application Message Consumption Scenarios

In simple terms:

  • Scenario 1: All the services will be listening to the only available queue and the message distribution will be round-robin,
  • Scenario 2: Each service will create its own queue(programmatically also possible from the service) and the service will only listen to its queue. If you have let's say multiple instances of Service A, depending on your business requirements, each service can consume the same queue, or create their own queues.

A Small Reminder; the queues must be initialized first, before sending the messages to the Topic, otherwise Messages will be LOST

In addition, always consider failure cases and work with Dead letters upon application issues. You would not like your messages to be lost in case of transaction failures. Here is an example:

https://zoltanaltfatter.com/2016/09/06/dead-letter-queue-configuration-rabbitmq/

Last of all, my suggestion (considering you are developing your application in Java and using the Spring framework) The Spring Cloud Stream Framework. As usual, Spring lets you focus on your Business Logic rather than Engineering and re-inventing the wheel. On top of that, the framework offers any of scenario above and error handling as well as retry capabilities. Please consult below resources:

| improve this answer | |
0

I have recently been working on the very same thing, using Rabbit MQ as an event bus between microservices. I am using Topic exchanges, where each publisher declares a topic and publishes different events to the topic, using the routing key to differentiate event types. Then, each subscriber declares a queue bound to either a specific routing key on that topic (if they only want certain types of events) or bound to #, which is a Rabbit MQ wildcard that lets you subscribe to all events on a given topic.

I find it useful for each subscriber to at least have a different queue for each topic, as I think it makes it easier to monitor and keeps a large burst of one type of event from slowing down all other events for that consumer. I did make all of my publishers and subscribers self-wiring, that is, they auto-declare and bind their topics/queues on startup so there's no manual configuration requried. This seemed to be the easiest way to do it and was fairly simple using the Rabbit MQ client QueueDeclare and QueueBind functions.

| improve this answer | |
  • Thank you for your fast reply. A couple of question, do you set up a list of predefined services names to iterate over on startup? I mean, you declare cars, notifications and trace in a list and when other microservice startup creates queues like other-ms_[NAME] and binds each one with every exchange? And/Or instead you list the available exchanges from rabbit to build up that list? In addition to this, I suppose that for creating the topic-specific queues you iterate over the mentioned list and a list of well-known topics that you want for that microservice, Is it the case? – Sergio R.B. Nov 18 '19 at 23:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.