How to design inter-service communication with notifications microservice?

Question

I'm creating an application based on the microservice architecture and stuck on notifications microservice design. Basically it should be a service that notifies users about events happening in the app through multiple channels. Channel selection should be performed based on user preferences for particular notification type or maybe notification category, not to make it too specific. So, I decided to make notifications service responsible for managing user's notification preferences.

I use Kafka for inter-service communication and here comes the problem, I'm not sure how other services should interact with notifications service. I came up with two approaches:

1. Every microservice owns some topics. E.g. User microservice owns user.account.email-update, user.profile.update, etc. Orders microservice owns orders, orders.cancelled, etc. No microservice produces to other's topics (single writer). Notifications microservice keeps a list of topics it should consume. Obviously, messages in this topics have different structure despite some common fields. This makes notifications service responsible for transforming heterogenous messages to some generic format that can be understood by services responsible for delivering of notifications through particular channel (SMS, messenger bot, email).

Pros:

• Services produce messages once.

Cons:

• Coupling? Notifications service should be aware of every topic's message format and how to process it. And there will be a lot.

2. Services responsible for producing messages also to notifications.* topics. E.g. notifications.user.account, notifications.user.profile, notifications.orders. This allows to generalize message structure to some extent.

Pros:

• Some message structure generalization.

Cons:

• Multiple root messages for single event.
• Shift of responsibility for notification initiation to individual services?
• Coupling again?

Notification delivery services incapsulate assets such as simplified user representation and templates. They should be able to select appropriate one based on event type, and extract notification context and provide it to renderer to get a message to send.

Both approaches look tightly coupled to me and doesn't look scalable. How can I improve this architecture to make services more scalable and maintainable?

Also I'm not sure about a separate topic for each channel, and message structure. If I go with a topic per channel, it seems logical to include recepient's email as a field into a message sent to notifications.email topic. This makes notifications service responsible for fetching user email by ID. The same logic is applicable to notifications.messenger topic and chat ID. And this can go out of hand quickly. Due to delivery services' having of their own simplified copies of user table, I probably shouldn't include these fields. This leads to another question: do I need separate topics for channels at all?

Answer 1

There seems to be a lot of over-engineering going on. I, of course, don't know all the requirements here, this is just my initial impression.

Using Kafka as a generic communications medium is a bit cumbersome. Every sender-reciever pair likely needs its own channel, configuration, sizing, etc. Unless you have a very good reason you need to use it, like very large and lumpy call batches, I would go with good old HTTP, which is much easier and more transparent.

Remember, you can always add intermediaries later. If you don't need this immediately, even if you feel you'll need it later, just add it when it's actually needed.

About coupling, and messages. Try not to focus too much on technical things. When designing microservices, boundaries, even objects, it helps to think about what do you want to achieve. I found it helps to think about these things like a person, someone familiar with the company and processes.

If I would think about the "messaging service" as a colleague, I would tell it: "Hey, please notify this customer that his card expired.". I would expect my colleague at this point to look up who that customer is, how this customer likes to be notified, gather the necessary phone numbers or email addresses, and then craft the message appropriately for that channel.

Microservices (and even objects actually) should work not too dissimilar to this, as far as possible.

This will result in decoupling. Not micro-managing parts of the system will leave those sub-systems to evolve and get better in their jobs.

My answer may be a bit meta, but HTH.

Answer 2

There are likely to be two types of User Notifications in your system.

• User specific notifications - basically notifications that go to 1 (or at most 2) users.
• Broad notifications (those going to all/a group of users).

As such I think that your "Notification Microservice" should handle each case separately.

User Specific Notification

I don't think it makes sense for the NM to have the same logic repeated over and over again just to convert a very specific event from a given service to a user notification.

Instead, it makes sense for all the other services to generate a standardized (single) "User Notification" message. This might be parameterized so that the NM can pick up a template and substitute values etc. This message would be dispatched to a specific topic (multiple writers, one reader).

In order to make this work it might be necessary for standard services to dispatch two messages:

• A user notification message.
• Their own custom message for other services to use.

The Notification Microservices responsibilities are

1. Monitor the (single) User Notification Queue.
2. Identify if a user wants a particular notification.
3. Identify the correct template.
4. Do variable substitution.
5. Dispatch to the correct notification channel (SMS, Email, etc).

Broad Notifications

This is your first architecture diagram. Each standard service generates a custom message (used by multiple services - one writer, multiple readers) that the NM translates into notifications to multiple users. There are several reasons I think it makes sense to use this architecture for "group" messages:

• There would be a lot of repeated logic interrogating the NM about preferences in order to figure out who messages should go to.
• And/or there would be a bunch of messages sent to the NM that later get dropped.
• There is no need to have looping logic generating multiple messages in each standard service.

So for these messages the Notification Microservice does need to have detailed knowledge about multiple custom events (and monitor multiple channels to pick them up).

Summary

If you build a client library that can dispatch the "single user" messages to the NM topic it should be quick and easy to integrate that into the standard services and that should handle the majority of your notification needs.

For the more complex cases it makes sense to write custom logic in the NM to handle them.