In a microservice architecture, it's commonly admitted that each microservice should have its own replica from another "main" microservice acting as single source of truth. This keeps microservices autonomous and loosely coupled.
When the "main" microservice changes, it emits some events, so that interested microservices update their own replica and keep in sync. By using a Broker with persistent queues, we can "quite" ensure that no events get lost and replicas keep up to date.
But "quite" is not 100%, and there still are other ways of getting ouf of sync :
- If a newly developed microservice joins the system, how does it build its replicate (since all events have already been emitted) ? Should we introduce again some kind of (anti-pattern) sync communication to query the whole data from the "main" microservice and build the replica ?
- When should this process take place ? At the system startup (on a device) ? Periodically on a cloud architecture ?
- Should we block the whole system until this synchronization is over (we don't want the "main" microservice to emit new events while we are synchronizing, because we get the risk to get out of sync again)
How did you solve these problems on your implementations ? I've seen somewhere the concept of "reconciliation" but I did not find any implementation of that concept.
Many thanks !