How to define overall monitoring of event-driven architecture?

Each component talk to other component via events. Consumer(s) react to events within minutes to days based on use case. Each component performs some work, updates the database and generates events. Event consumer then performs some work and updates the database. Database here is dynamoDb. I am persisting all my data in Elastic search as well. I have millions of records in DB.

For instance: Component A updates record X in database and generates an event to be consumer by Component B which has to reach to event within 5 minutes and update the same record X in database. How do it monitor that A is generating event and B is listening to it ie. X is getting updated within 5 minutes Alarm should trigger if X is not getting updated within 5 minutes.

(I have use cases where record need to be updated within seconds, minutes, days and weeks).

  • It could be a design smell that more than one service updating one persistent object.
    – ivenxu
    Jul 16 '18 at 23:59

As you already are sending events between producers and consumers, it should be relatively easy to add another consumer for the events.

Depending on the complexity of your system and the alarm rules, you could have a global monitoring system that subscribes to just about all events in the system, or you could have a separate monitoring system for each alarm, which just subscribes to the events that are relevant for that one alarm.

You might find that you need to add some monitoring-specific events to the system, for example an event from Component B that it has reacted and updated the database, so that the monitoring component doesn't have to poll the database.

Both the correct functioning of the monitoring component and the correct relation between sending (monitoring) events and database updates should be established using (automated unit/component) tests.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.