3

I'm working on a complex system that follows the principles of microservices, but with one exception - my services are more 'generalized' and big (I refer to them as macroservices further down the post).

For example, instead of having very precise services:

  • users
  • news
  • articles
  • posts
  • threads
  • matches
  • items

I have the following generalized services instead:

  • portal (users, news, articles)
  • forum (users (from portal), posts, threads)
  • game (users (from portal), matches, items)

I understand that it affects scalability, flexibility and coupling - but this design choice has its reasons, and it's not the problem I'm facing either.

The problem

As you can see above - all of the macroservices use users resource originating from the portal service.

Generally speaking, once you register on the portal - you can use the same account to play the game or post on forums.

Question:

Should I implement cross-service communication, so game and forum fetches user from portal when needed (quite often, as every post, thread and match has user(s)) - which means they can't function once portal dies, but the data will always be up to date.

OR

Should I implement data duplication/replication, so whenever UserCreated, UserUpdated event fires - game and forum stores a duplicate of user in their own database - which means they can still function if portal dies, but there's a bit of coupling due to synchronization.

Improve this question
2
  • 1
    I'm a little bit confused. From your example use case, it seems you are looking for a Single Sign-On solution which won't need data duplication/replication. Is there any use case where you actually need the users data in all three of the macroservices, as you call them?
    – Bogdan
    Sep 20, 2020 at 14:24
  • @Bogdan Yes, it's a little bit more complex. I have to fetch users for forum threads and posts, as well as game match participants - and it's all the same, shared resource: Portal.Users
    – JCode
    Sep 20, 2020 at 14:32

1 Answer 1

Reset to default
3

Since we established in the comments that you need more than SSO, which would have been the simple answer, for sharing the data between the three "macro"services the answer is a little bit more complicated. And it's more complicated because the answer can only be "it depends".

Both your approaches have advantages and disadvantages. Fetching user data from "portal" every time it's needed inside "forum" and "game" will allow the data to always be up to date but might cause performance issues or slow response times of the applications while you constantly fetch data (and if "portal" is down, so are the other apps). Data replication with update events can cause the services to see different data at the same time if some event is lost or replication hasn't yet propagated all the data. You now have eventual consistency and data in three different places. Change the schema of users in "portal" and now you need to make changes in two other places.

The answer is "it depends" because the solution you choose (one of those two, or some other) heavily depends on your use cases. So this is the first thing you should do. Look at all your use cases and see which solution would work best in each case.

With that being said, and the fact that you are dealing with user entities, which shouldn't change all that often, I would personally go with replicating the data. You have various options and tools available: you can do an initial dump of users data then subscribe for changes, your RDBMS (whatever you are using) might be able to handle the replication in a master-slave configuration, use a lighter NoSQL solution to distribute data, a CDN even (depending on what data you need to replicate), etc., there are plenty of options. Be very careful though to keep the replication as decoupled as possible as things like databases often introduce a very tight coupling between services ("micro" or "macro" as they may be).

If users are aware that changes to their data will take some time to propagate, it won't be an issue.

Improve this answer
10
  • 1
    This makes perfect sense. With replication I can also have different columns, except for the ones I want to replicate from portal, like id or nickname. Do you suggest creating a subscriber/listener system to listen for user updates or a synchronizer service?
    – JCode
    Sep 20, 2020 at 17:05
  • 1
    With a pub/sub you can apply changes as they happen, but you can also miss events. For ex, if someone changes user data in "portal" while you deploy "game", you will miss those events in "game". You can replay the events, but that complicates things. So it might be simpler to run a synchronzation service later on (every night for ex) to reconciliate the data and make sure everything is up to date. If you write the synchronization service in a smart way (like only sending recently updated users, not everything) then you might run it more often, depending on your use case, even every few minutes
    – Bogdan
    Sep 20, 2020 at 18:31
  • 1
    it sounds like if I built it as a monolith - I'd have 100% data consistency, but not 100% reliability. What if I implement a queue that can re-run timestamped jobs if target services fail? Does that sound like a good solution?
    – JCode
    Sep 20, 2020 at 19:13
  • 1
    That might work. There isn't a right or wrong way to do it, as long as it's a reliable way of getting the data updated.
    – Bogdan
    Sep 20, 2020 at 19:29
  • 1
    You're not wrong. That trade off is inherent to all distributed systems
    – Nathan Cooper
    Sep 20, 2020 at 23:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.