2

I'm working on a distributed application and getting deeper into Event-Driven architectures with Microservices. Let's say I have been running two Microservices, CustomerService and AccountService. They both have their own database, and they've been keeping in sync with an event-style messaging with RabbitMQ in between.

For example, when a new user is created in the CustomerService, a CustomerCreated event is sent to RabbitMQ. The AccountService has a subscriber that listens to this event and immediately creates an account for this user.

Now, several years in the future, I want have a new service, MusicListService, that whenever a new CustomerCreated event is raised, bind a Music playlist to that user. Obviously the previously created users need to have a playlist binded to them as well. But since this service is brand-new, it didn't receive the past CustomerCreated from the former years, so no playlist will be binded for the former users unless I create some kind of out-of-band script that brings the database to a proper, up-to-today state.

What would be the best start-up strategy to a new Microservice so that it keeps in sync with the ones that were already existing?

  • 1
    It may not solve this issue but you should look at Kafka. It's structured in a way that eliminates a lot of messaging challenges. For example, if you were keeping all events (years might be tough) there would be no extra work to handle this. All the events would be available to the new subscriber. – JimmyJames Nov 1 '17 at 13:58
2

Generally you take a snapshot in time of existing data and either process that snapshot via an external process or by replaying the snapshot’s messages over the bus/queue.

Practically, you’ll need to account for out of order messages, as well as duplicate messages because the snapshot and normal message stream need to overlap a little to ensure there’s no gaps (and thus missing data). But you should already be accommodating for that since you’re using a messaging system.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.