MQ Architecture: How to handle old messages in the queue and releasing a consumer/producer upgrade

Question

So for example if I have a producer that sends a message

{id: 1} 

to a consumer that expects that json object. And there's a ton of messages sitting in the queue waiting to be processed. I'm also writing some code in the producer & consumer so that it accepts {project_id: 1} instead of {id: 1} now. This will cause all messages currently sitting in the queue to be rerouted to the dead letter queue.

Is this generally bad practice? Or how can I assure that all the stale messages are processed before releasing an upgrade.

Answer 1

We try to make sure all changes to a message contract are both forward and backward compatible. In those rare cases where there is truly a breaking change in the message contract, we consider that a different message (a new kind of message).

We use the Parallel Change pattern to manage such breaking changes.

In cases where we have control over the downstream consumers, we make sure that they are ready to consume the new kind of message before it starts being published. It will also retain it's ability to consume the old kind of message so it can drain the queue of any existing messages. Depending on your messaging platform you may or may not need to use two queues.

In cases where we don't have direct control over the downstream consumers, we publish both the old and new messages in parallel. This gives the downstream consumers a grace period where they can migrate away from the old messages in favour of the new ones. We flag the old messages as deprecated during this period. After the grace period, we simply stop publishing the old messages.