In an event-driven microservices architecture, services typically need to update their domain state and publish an integration event to a service bus at the same time (Either both operations complete, or none).
Simple example: The ordering microservice receives a REST API call to create an order. The microservice needs to:
1) Create the order entity and persist it in the database
2) Publish an OrderCreated integration event through an event bus
When using a relational database this is typically achieved using the outbox pattern: a single entry is saved in the database that indicates that event X needs to be published. This entry is saved as part of the same transaction that contains the domain-state changes. A background process then polls these entries and publishes the events. This means that the event will eventually be published, making the system eventually consistent.
However, NoSQL databases do not favor the idea of updating multiple documents in a single transaction, and many of them do not support it without ugly workarounds. Below is a list of potential solutions (Some more ugly than others):
1. Outbox pattern variation:
Outbox pattern, but instead of having a separate collection of documents for the pending events, they will be saved as part of the domain entity. Each domain entity will encapsulate a collection of events that remain to be published and a background process will poll such entities and publish the events.
If the background process publishes the event but fails to remove it from the domain entity, it will re-publish it.This shouldn't really be a problem if updates are idempotent or if the event handler is able to identify duplicate events.
- Domain entities are corrupted with integration events.
2. Event sourcing:
Event sourcing makes this problem go away but is very complex to implemented and a big overhead for small microservices.
- Complex, might need complete re-design of the way services work with data.
3. Listening to own events:
The service will only publish an event that is also subscribed to (It will not update its state as part of the same operation). When the service bus sends the event back for handling, the service will update its domain entity.
- Other microservices may handle the event before the origin microservice. This may cause problems if they assume that the event already happened when in fact it hasn't.
Are there any other solutions to this problem? Which is the best one?