2

Let's say you have the following application: users can be part of different organizations. Different users can create tasks and projects within an organization. In a microservice architecture, there are three separate services that manage tasks, projects and organizations respectively.

Now,let's say we want to send a daily email to all the users that contains all the new and completed tasks and all the new and completed projects within an organization, a sort of daily feed.

What's the best approach to build out this feature? I feel that using the public apis won't scale well, especially because the emails are generated all at once, and it could potentially be a lot of requests, linear to the number of users and organizations. Another approach is to have the "daily email" service subscribe to the task and project events, store them in its db and then flush the db when the email is sent (this can be easily mapreduced down the line). Is there a disadvantage to this? Are there other common patterns?

2
  • Can you be more specific in what you mean by "won't scale well". A lot of requests isn't in of itself a bad thing (Google handle billions of requests a second across their network). What is your specific concern about scaling? Are you concerned about the cost of the hardware to scale out to handle lots of requests Aug 4, 2016 at 10:03
  • You can split it further: Daily email service & (change)log service. Then you can re-use that log service also for other purposes. The idea you have with events, if you already have the structure, sounds good. Because you can re-use those events and it provides decoupling. So event goes to log, daily mail requests log and users service. And combines that data and outputs e-mails. The sending part can also be extracted so your daily email service creates the mails. Another service (even external) handles queuing, re-deliveries, error handling etc. Aug 4, 2016 at 12:56

2 Answers 2

3

What's the best approach to build out this feature? I feel that using the public apis won't scale well, especially because the emails are generated all at once, and it could potentially be a lot of requests

One thing to pay attention to is that it isn't a lot of requests, but a lot of queries, which is to say reads. In a use case like this one, where you have lots of interest in the same data, caches can be very effective.

Another approach is to have the "daily email" service subscribe to the task and project events, store them in its db and then flush the db when the email is sent (this can be easily mapreduced down the line).

In , it's common to serve reads from pre-built "projections". The main advantage you get from something pre-built is improved latency, rather than scaling.

One consideration is whether or not your domain events should contain enough information to build out your projections. Udi Dahan talks about services being a technical authority; among other things, you get better decoupling between your services if you don't let "private" state leak across the microservice boundary.

So if you are thinking of your email publisher as a discrete microservice from your projects/tasks service, then the events published would include the event type, and the identifiers for the involved entities, but not any of the internal data.

<TaskCompleted id="12345"/>

This approach understands composition slightly differently - the subscribed microservice delegates the presentation of the data to the technical authority, rather than duplicating the raw business data between two different services.

On the other hand, if the email publisher is part of the same service as task management, then maybe that restriction doesn't apply.

1
  • That's a good point about notification only having the proper ids and not the content. Aug 7, 2016 at 15:32
0

Another approach is to have the "daily email" service subscribe to the task and project events, store them in its db and then flush the db when the email is sent

My first thought was to query your services via their API. However the above looks good if you have such events. You don't have to extend your services to support or optimise queries particular to a report, you won't put additional load on your existing microservices, and you can extend the above pattern to multiple reports.

As such it seems a very suitable approach.

Maybe you don't want to flush that db, by the way. Maybe you want to hold onto that info for a given period of time, to cater for audit, outages etc.?

Perhaps my only concern with such patterns is the very loose coupling (i.e. you're listening to events, and what happens when your services miss events due to misconfiguration or similar?). You can mitigate that with suitable testing, monitoring and configuration management (e.g. deploying all services with the same configuration set such that you can't configure service A differently from service B).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.