Service bus and strict message order - how can I handle failed messages?

Question

I am considering using Azure Service Bus, however my question applies to queues in general.

I have a requirement to send messages in a certain order. Say that the publisher sends a message to the queue, and the subscriber receives the message, but it fails to process the message for some reason (e.g. some business rule is broken or the subscriber calls a downstream service that is down.) In that case, I cannot send subsequent messages because it would be out of order and the downstream service will return an error.

So in other words, if the subscriber fails to process message 2, I must not send message 3.

My question is - what sort of pattern can I implement to address this scenario?

Answer 1

If you want this kind of backpressure (i.e. the publisher grinding to a halt if the subscriber is not working for some reason) you should ask yourself, whether a Service Bus is the correct tool for your problem. This sounds very much like a synchronous type of operation, which maybe easier to implement with a different type of communication.

Nevertheless, as candied_orange already mentioned, you can wait for a confirmation and handle the in-order requirement on the producer side. (In EIP they call this the request-response pattern.)

On the other hand, you can take the decoupling to the next level, and guarantee ordering on the consumer side. Only consume the next message in order, or even hold an internal ordered queue for messages you still have to process. This way, the publisher can happily continue working while the consumer is down or has some kind of other problem.

If both strategies don't really apply, ask yourself why you have two different services anyway. The much-hyped microservices today don't necessarily mean that you should separate every aspect of your application, but that you should find useful separations. And two services which have to work in perfect synchronization are mostly not a useful separation at all.

Answer 2

So in other words, if the subscriber fails to process message 2, I must not send message 3.

Simple, don’t send 3 until you’ve confirmed the successful processing of 2.

Unless you can build some transactional logic into the processing you have to enforce any such rules yourself.

All of which leads me to question why 2 and 3 are separate messages in the first place.