0

I'm working on the specification for building a web services platform and AMQP is the messaging protocol used for the interaction between the components of this platform.

I'm using topic exchanges for this, and I want to define a common approach + some naming convention for the routing keys.

For these topics RMQ exemplifies/recommends using stuff like: lazy.orange.elephant quick.orange.male.rabbit topic exchange - RMQ doc

Now,I have a colleague that recommends using the name of the destination component into the topic for the cases where the message is only intended to be processed by this component. E.g: orange.orange_juicer_01, orange.ice_cream_machine_02

Does this look to you like an anti-pattern?

3

It seems like a mistake to me. The point of publishing a message is that you are decoupled from the component or components receiving the message.

The business logic about whether the message is then fanned out or routed in whatever way is outside the scope of the publishing component.

But there is obviously a connection between the publisher and consumers of the message in that they both know the topic name. ie If you only have a single topic you have to listen for all the messages. So the choice of name is important.

My feeling here is that your colleague is conflating topics and queues. It's good for the component to have its own private queue. This allows it to run without bumping into other components. But topic names need to be public and unchanging so that you can insert components into your overall flow.

So in your case I would have topic 'orange' and queue name 'juicer1' When the juicer component starts up it would create its Unique queue name and route all/some/fanout/round robin 'orange' messages into it. Then start pulling them down from its private queue to be processed.

Complex routing and 'MasterWorkers' in order to achieve the more complex routing you require you will have to program that routing into a component and set it up a bit like this

Worker -> starts up and contacts master worker "I'm ready for work! here is my queue name"
MasterWorker -> remembers the workers queue name
incoming queue -> pulled by MasterWorker
MasterWorker 
    -> do I have any Workers available? 
    -> does the avaialble worker match business logic x?
    -> send the work to the workers queue
  • Awesome answer @Ewan thanks! Our idea is that each component here defines its own private queue, and we define "conventions" on the topic names (so they know how to pub. and what to subscribe to). Now, imagine if the publisher -which generates oranges- wants also to play the role of a dispatcher, e.g, it knows there's already too much orange_juice out there, so it want the ice_cream_machine to process the orange event. I guess we are here in the borderline of how a publisher and a consumer SHOULD be used and what topic exchanges are meant for, but what would you suggest doing? – fedesismo Nov 17 '16 at 9:17
  • yes exactly, you have grown beyond the limits of the default behaviour. (which in my view is very limited) Just throwing stuff into a shared queue and pointing multiple workers at it only works with the most basic scenarios. If you want to have even some simple management, say "how many workers are running" you need to create what I call a "MasterWorker". – Ewan Nov 17 '16 at 9:55
  • edited. i did do another answer to a similar question in more detail but cant find it now – Ewan Nov 17 '16 at 10:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.