Right now we have a set of micro services with a client-facing gateway responsible for authentication and routing. MySQL database and RabbitMQ for the message queue. Internal services may queue up requests as necessary but the gateway never does. What I’m finding is that the time for the gateway to respond is too long. I want to queue messages from the gateway to async the entire system and generate faster response times but I’m not sure how to guarantee persistence so we can handle failure. I’m the absence of an actual database record I’m not sure how it can handle proper persistence.
In short: how does a fully event-driven architecture guarantee persistency of data. It will be easier to elaborate with a concrete example.
We collect earning records and then later allow people to make payments based on their earnings. We have an endpoint to create an earnings record. What it looks like is the following:
client -> gateway -> earnings service
The problem with this is that it’s all synchronous. The earnings service call takes too long to do all the work. It has to write some records, do checks to make sure the data is valid, etc. There are other asynchronous portions but not this part.
If I queue at the gateway then there’s no database record yet. The first record gets written in the earnings service. In our synchronous system that’s fine because we guarantee a record on a 200-level response. If I start queueing at the gateway I now need to ensure we don’t lose messages without a db record. How would I do that?
Is there some sort of way to guarantee persistence in Rabbit? Is a database of messages an option? Our endpoints are idempotent so in a disaster recovery situation we can replay messages without harm. I’m just not sure how to go about doing this. Thanks