My architecture looks like:

enter image description here

Each block is an individual service. There may be more upstreams and downstreams added in the future. The upstreams need to send email notifications to users. Each email is specific to a single upstream.

The question: Where should I put the code to call Email Service, in upstreams or downstreams?

I'm thinking the ideal obvious choice is in the upstreams, as the emails are related to the upstreams, not necessarily to the downstreams. Also, a single downstream endpoint may be used by multiple upstreams, so would be messy to add email logic for multiple upstreams in a single downstream endpoint.

However, in addition to upstream requests, there are background emails. For example, if something hasn't "happened" (determined by downstream) in 1 hour, then send an email. These are driven by cron jobs and queues. It would be nice to place this background email logic in the upstreams as well, however:

  1. Some of these queues must be consumed by downstreams for reasons unrelated to email. The downstream can produce to new queue consumed by upstream which sends email, but that's more engineering effort.
  2. Some downstreams already send background emails triggered by cron/queues, which would require migration to upstreams. Again, more effort.

Right now I'm thinking the approach to be:

  1. For emails triggered by upstream requests (synchronous flows), upstreams own the email logic.
  2. For emails triggered by background (async flows), downstreams own the email logic.

I feel this is the best tradeoff between design and effort.



1 Answer 1


Follow the data

First what are you trying to achieve? You are wanting to consume some data X and apply some behaviour (format and send an email). It doesn't matter that you are sending an email, this could just as easily be a report, or any other IO device. So lets zoom out and look at the data flows.

You will find that each behaviour needs different data. Like you've pointed out some need data, but also the knowledge that time has passed. Others need data from several different processes. The point is that each of these output devices are linked into some set of data, delivered through this or that path.

So the answer becomes the behaviour should be where the appropriate Data flows converge. If that is the upstream program, downstream program, or some other place then the most logical place is that place.

That being said, it isn't always the best idea to keep growing a process to have more responsibilities. It tends to lead toward brittle code. It is better to introduce a new data-stream and plugin the new output processors. Those processors could be housed in a dynamically linked library (loaded into the same process), or across a network/IPC boundary (in another program).

Another note is that you can have considerable control over where those data-flows converge, there are downsides to pushing flows around, but they might be offset by other convinences.

  • Data flows converge - could this be rephrased as the "driver of the flow"? So emails would be sent by upstream for synchronous flows (since upstreams drive the flow from user) and sent by downstream for asynchronous flows (since downstream drives those flows using cron or queues).
    – onepiece
    Jun 25, 2021 at 8:26
  • @onepiece Potentially. I'd call a "driver of the flow" a process though. Data flows through many processes, and those processes often take more than one input, and/or produce more than one output. The processes are nodes in a data graph, Their inputs and outputs are edges on that graph. A logical place to put any functionality is where there is a process with access to the necessary inputs, and controls the desired outputs. Of course this might not exist and require the creation of a new process, or redirection of an established data flow. (Part 1)
    – Kain0_0
    Jun 25, 2021 at 9:00
  • @onepiece You could in theory create a flow of data from all of these systems to a process explicitly for notifications. You could place the email responsibility in each process that fits it best. Both have trade-offs. In the first you are making a very large process and establishing new data flows, but you aren't changing the existing data flows. In the second you are complicating the processes in the data paths, and potentially direct some data to a process that wouldn't otherwise need it.
    – Kain0_0
    Jun 25, 2021 at 9:03
  • Yep your second comment is exactly what I'm struggling with. Approach 1 means placing email triggers in upstreams, Approach 2 means downstreams. Approach 2 is less work, but there is less separation of concerns and may be less sustainable as the system gets more complex. Generally, which approach would you go with?
    – onepiece
    Jun 25, 2021 at 18:35
  • @onepiece Generally? It depends. Not to sound flippant, but: the size of the process, where the process is, how valuable it is to have the emails centralised, local resources, existing infrastructure, and target state all have a voice in what i'd generally do. Given that you already have a decentralised system, i'd be keeping it that way with each process responsible for emails on their data sets (within reason). But i'd still want good modularisation by putting the email logic in a plugin for notifications/reporting. Making it easy to grow/change emails/reports/etc...
    – Kain0_0
    Jun 26, 2021 at 4:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.