My project has a integration with a external system. We need to send some important informations to this system. For this, we create a micro service to connect to this external system. This micro service receive async messages from our internal systems in a dedicated Queue for that, and the micro service read the messages and try to send them to the external system. Very simple.
Before explain my doubt, I will put you guys on the context.
We are not sure if this external system is reliable. Sometimes the external system will be off and we need some way to recover from this. So we implement a retry mechanism. But we also imagine that this external system could be off for a lot of time and maybe even reject some valid messages until we talk with the support team. So, we also create a durable Dead Letter Queue to receive this rejected messages.
In the worst scenario, it will be a lot of messages on the DLQ queue to analyze the cause of why they are rejected. So my team have the idea to pull the messages from the DLQ queue and persist them as a Json in a database, because we imagine that will be necessary to edit them.
And my question is about this last paragraph: save messages on database. I'm not sure if is a good idea, seems to me that we are just replicating the durable feature from the Queue on a database.
The idea came from the fact that the messages on the database will be easier to analyze (maybe we will need to do that), edit them (maybe we will need, probably not) and have a more reliable way to manipulate the messages, because one mistake on the RabbitMQ Management and the message is gone. We will also will need some kind of cron job just to read this messages from the database and send them again for the main Queue.
As you see, there are a lot of maybe and a lot of extra work (database, tables, cron job, etc), and I'm not really comfortable with the solution. Of course we will be more safe if the messages could be analyzed and edited on a relational database, but I'm not sure if we will use this feature on the future.
I made some research about the use of RabbitMQ for rejected messages that could be analyzed for the team and sent back again to the main Queue, but this is a not a common scenario.