10

I would like to create a system which handles alert messages from various programs and can process those alerts to down-wind consumers via email. This would all be contained over one internal network.

I think I want the basic architecture to look something like this:enter image description here

The main concern I have currently is the "Message Handler" bit, which is what will be my "sort-of-API". I want all components of this system to send data to the API, which handles all writes to the database. I think this approach is easier because it simplifies security, and allows me to contain a lot of the more complicated DB queries into one single program.

The concern is that I want this to be language agnostic - meaning that any code should be able to send messages to my Handler - which will interpret them. I hope to do this via JSON flat files - or via REST calls to the program (giving flexibility to the down-stream applications).

My question is-

Should I bother with the message handler - or would it add simplicity to just allow direct database access to the down-stream applications, as well as the two other components (Management Console, and Alert Manager)?

That way, they can insert whatever alert they would like - as long as the INSERT into the DB table/s is valid.

I'm not a software designer by trade so excuse me - I just want a project to do in my free time.

4

Have you looked into AMQP (Advanced Message Queuing Protocol: https://www.rabbitmq.com/protocol.html)?

RabbitMQ is an awesome tool for something like this, I think (there's others as well, MSMQ, Azure/AWS services, etc.). Not only do you get one language agnostic message handler (simple "send the message to the message server w/ json data"), you detach the downstream message processing and make it well isolated. Run a message service that processes incoming messages from the queue(s) you need and spit out your notifications.

One of the reasons I really like using AMQP is that you start off like you are now with some home-baked solution, but realize after time you need to handle messages slightly differently depending on the type, who it needs to go to, etc., so you end up essentially building your own AMQP implementation anyway.

What do you do if a message needs to go to 5 different recipients? What if you have a message that should be rotated throughout a number of processors (think long running tasks and having X number of simultaneous processors, where you can round-robin messages of a specific type). What if the message should go to one person, but if they're not available/online, it should go to another? The AMQP handles all of this (quite nicely!) already, with very nice categorization, queues, channels, durable persistence, all sorts of features.

Here's a basic overview of the scenarios it can handle (note this is not specific to RabbitMQ: it's an AMQP thing, but RabbitMQ happens to explain it well) - https://www.rabbitmq.com/getstarted.html

  • I have seen RabbitMQ implemented for other software before - always seemed interesting (if not a bit complicated). I'll take a look into it! I think I shyed away from this at first because I wanted a lot of flexibility to offer the data senders. So if it would be hard to implement a REST call onto an existing Powershell or Javascript, or heaven forbid a VB6 app, they can usually output to a flat file pretty easily. But being able to just replace a lot of the message handling would shave time and effort for sure! – Christopher Mar 30 '18 at 19:33
  • 2
    I was a little intimidated when I first got into RabbitMQ, but after a weekend of study it was like wow, this is actually really, really nice, and now that I understand what I'm looking at, it's actually really simple – jleach Mar 30 '18 at 21:18
  • I like the idea of AMQP, but the way RabbitMQ handles the client API's for each language sort of defeats the whole purpose of using a language-agnostic protocol. What if I have a client who is running a language that does not have a supported API written for it? Some of these features are really nice... but overkill when all I need to do is get 1 message from point A to point B, with nothing in-between. – Christopher Apr 3 '18 at 14:25
  • I was thinking to slap a quick rest API on top of it, which would cover your agnostic concern, but agreed that if you will never need any of the features that AMQP offers, it would be overkill (apologies if I had you run a wild goose chase on it...) – jleach Apr 3 '18 at 14:38
  • Yeah - a quick REST API to cover it would work.. but then I might as well just make a REST API of my own. And no apologies required - it is a great technology and learning is essential to progress :) if not now - I'm sure it will come in handy in the future. – Christopher Apr 3 '18 at 15:25
3

Very well framed question!

So- all architectural decisions involve tradeoffs. If you are curious for a discussion of tradeoffs perhaps edit your question in that direction. Instead, since the question just asks for a position, I'll take the side of arguing in favor of the MessageHandler. I will go a step further to suggest NOT including a database- at least not a SQL database, at least not to start. Just have the MessageHandler save JSON to the file system, say a directory-per-hour-of-received alerts (depending on volume, of course), and have the API when queried by the Alert Manager just traverse over the last 2 directories of alerts to decide what emails to deliver (depending on priority, of course).

There is a ton of good stuff to chew on in this problem, and keeping a database out of the picture at the early stages will remove a lot of incidental noise and unnecessary problem solving. Of course, perhaps you have a hidden love of creating relational data models and dream of writing SQL. In that case, this answer is totally wrong. But generally speaking even the most agile databases are terrible application platforms, and they only get included in systems because they're specialists at durability and indexed query.

Good luck!

  • 1
    I just like the ability of being able to control email groups, members, and various other nice things about using a Relational Database. The database design is really what I started with in the project - so I never even thought of removing it. Thank you for that input and point of view!! – Christopher Mar 30 '18 at 17:20
  • I knew it! :) Understanding what you're really looking to get out of something is the most important thing. Cheers! – Jonah Benton Mar 30 '18 at 17:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.