0

I would love an opinion on system design in a distributed system that operates under the same domain. I have a bunch of microservices following the Database per Microservice pattern, and updated via events where each database stores a partial representation of an object, based on what information they need.

Due to changing of requirements, some of these microservices will need to project new data into the databases (reconsuming all the events, updating the state in the db), therefore having to take responsibility for changes that are not always pertinent to that service.

I've been thinking about creating a microservice that contains the overall state of each object (like a redux store) and push Update events with the new state of the object, so that consumers, can update their own internal states as required.

What would be the downfalls of this approach?

2
  • 1
    This is just based on a hunch, but if I had to guess: It's a central point of load and failure. Each (stateful) microserver is essentially useless without this central storage, and you're probably going to be having a lottttttt of communication overhead, since every state change happens "at a distance"
    – Alexander
    Commented Mar 27, 2023 at 17:06
  • 1
    It breaks encapsulation. You might as well have a JSON interface with CRUD operations, or an ordinary Repository pattern using RPC. Commented Mar 27, 2023 at 19:32

1 Answer 1

1

I think this pattern is already known and called event sourcing.

The problem is that it's not easy to implement. You will need a resource to store all the events, thus have additional cost. And if you want your events to be consistent, you need a total order of the events, which is quite hard to scale. You can use partition/sharding to scale the throughput, but then it won't be as consistent (but, usually it doesn't have to be 100% consistent anyway).

Furthermore, there is a delay between the creation of the event and the projection. Imagine read-after-write scenario where you create a post and then refreshing the timeline. There is a chance that the db that supposed to calculate the timeline has a little delay and your post doesn't show immediately (even if you already see the "post sent" message).

The consistency can be problematic as well. Imagine you have a service that handle permission of images, and the service that store the images. You set the visibility to "only-visible-to-family" and then upload a photo. The service that stores the photo might receives the event first and the photo became available to all users (because the permission service haven't receive the event yet).

But, the benefit is you have all the events that happen in your system which can make debugging easier and if your database is gone, you can just rebuild it (theoretically).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.