Event sourcing and sync between write and read model

Question

I am creating this project based on event sourcing and cqrs. My write and read model are on different databases (and machines) and are connected through an event bus (in particular I am using MassTransit with rabbitmq for the transport)

When an event is generated it is stored in my write model and published on the queue where the read model picks it up and updates the db accordingly

What I still have not completely clear is how to keep the two models in sync in case of errors or problems

For example it could happen that the database on the read side goes down, or messages queue goes down itself and the write side cannot publish any more events

My first approach is, since every aggregate (I am using DDD) has a version number I can use this to keep track of how many events per aggregate instance I have processed.

For example, if in the read side my object with ID 8734 has a version number of 4, and a new event arrives for that object but the metadata says that now we are at version 6, the read notices it missed an event and it can use the message queue to ask for the missing one(s) (This method actually solves also the problem of processing events in order)

The drawback is that the system does not realize that an event has NOT been processed until the next one arrives. (If the event 6 for object 8734 is not generated, I'll never notice that I am missing the event 5)

A second option is: I can create a background processor that 'polls' the database and asks for new events, but this defeats the idea of pub/sub messaging and also forces the read model to process events sequentially, even if they are about different aggreagates where parallel processing would be permitted (and delaying the eventual consistency)

Lastly I could make sure from the read side I keep publishing an event until I receive the ack that the read model (or all the read models) have processed it correctly, but this shifts the repsonsibility on the write side of the application and I do not know if it a good strategy or it is going to bite me back later

How to deal with these situations?

Answer 1

Greg Young's Polyglot Data talk includes a discussion of pull vs push.

Pub/sub is "fine" when you are dealing with a single message in isolation. It's also a reasonable choice when you need to handle sparse events with low latency.

But pull based solutions are a lot easier to reason about when you are dealing with sequences of messages. The consumer always gets messages in the correct order, different consumers can use their own local cursor to track where they are in the stream, and to request the slice of events that they need to process.

The pub/sub mechanism becomes, in effect, a notification mechanism, to wake up any hibernating consumers (who can then poll the data to catch up to the present before they go back to sleep).

In February 2016, Raymond Rutjes wrote:

At the DDD Europe conference, I realized that the speakers I talked with where avoiding Pub/Sub whenever possible.

Greg and Yves Reynhout (among others) chime in.