1

Is it possible for a queue to have multiple consumers?

I am working on a system where a single queue needs to be accessed by multiple consumers for different micro-services. Is it possible for the queue to retain data until after a consumer has consumed it or are there other possible methods for multiple consumers to consume data from a queue?

2
  • 1
    Can you provide a bit more information? Do all the consumers require receipt of the message or is it a "first come first serve" situation?
    – maple_shaft
    Commented Jan 22, 2020 at 15:06
  • most of the consumers require same message i.e multiple microservice require same message Commented Feb 14, 2020 at 7:06

3 Answers 3

0

A queue should (except for specific cases) be treated as an inbox for each application, so each consumer interested on some message type should have its own queue.

Don't forget that AMQP promotes a producer/consumer pattern, so each app would "steal" a message to other, this is very nice when the consumers are many replicas of the same microservices, while if different microservices are interested in same messages they have to declare that want message copies on thier queues creating a binding. The binding is a rule that implements the following behaviour

given the exchange < E >

when a message is sent on < E > with routing key < k >

then put a copy on the queue < Q >

So the queue is just a buffer, the routing key resembles the content of the message

when the messages are put on the queue, the consumer receives is and executes some business logic. If the logic has been correctly executed, the service doesn't need the message anymore. If an error occours, you have to decide if there is need to retry the execution (hopefully with a different healthy replica) or give up and save the message for further post-mortem troubleshooting. This is accomplished with ack/nack packets (and the dead letter exchange) that the consumer sends to the broker. Most of AMQP libraries do the job automagically: Spring AMQP sends a nack if an exception has been thrown through the message listener associated to the queue

1

can a queue have multiple consumer ?

Yes it's possible that multiple consumers take messages from the same queue. But every message is only consumed once by one consumer

is it possible for the queue to retain data until after a consumer has consumed it

No the messages are not visible for the other consumers if one consumer started consuming it.

I am working on a system where a single queue needs to be accessed by multiple consumers for different micro-services.

I think you need a queue per cause/topic that has to be handled

1

Yes, but it's not a queue, it's then called the publisher/subscriber pattern.

Queue - every item gets processed once. If there are multiple agents, every work item still gets processed once. Example: Payment processing

Pub/Sub - every item gets processed by every client/agent. Example: Customer change of address.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.