0

I have an event driven architecutre, with many microserver that publish a lot of message in my event bus.

In some case i can't track why some process is blocked, maybe because my Service-A haven't publish the message in the bus after the transaction saved or maybe the other Servics-B is crash immediately after have take the message from bus.

My idea is to save on my database Service-A the record that indicate that event was sent successfully from my Services-A. In this way i can exclude that have the problem on Service-A.

It's a good way?

Thanks

3
  • 4
    Logging is good. Only tricky thing I would mention is: It can bog down the database. In fact, using a database (traditional) for logging could very well bottleneck your application. It's much cheaper resource-wise to write log files to the file system (if you can) and use any easily-available collector software to ship them off the system to someplace somewhat more permanent, where you can then inspect them, when needed, using any easily-available log-grokking tool. (Some companies make a business out of it, e.g., Splunk, but there are plenty of alternatives, free or paid.)
    – davidbak
    Mar 26, 2021 at 19:36
  • yes, i totally agree with you. Storage on database is very dangerous. Maybe i write the message on my log system is the best way.
    – FedeC87p
    Mar 26, 2021 at 19:42
  • You should publish event as part of the transaction. That way, you will never be in in the situation above (trans saved / no publish.) Are you using a framework (such as Eventuate?)
    – Nate T
    Mar 26, 2021 at 20:47

2 Answers 2

0

You may be using your message queue incorrectly.

Most message queues allow you to specify the level of reliability that best fits your use case:

  • Sometimes, it's all about fire and forget, and it is perfectly acceptable to lose messages from time to time. For example, if you collect statistics to use for monitoring purposes, it's not a big deal if you lose some messages containing the disk, CPU, or memory usage—if you start losing them for a long period, staff at NOC will realize eventually that something is missing, and if you miss one or two from time to time, nobody would care.

  • In other cases, it would be unacceptable to lose even one message. For example, if a message indicates that a customer paid for a product to be shipped, you'd better be sure the target system received this message, or you'll have to deal with an angry customer on a phone a few days later.

Most scenarios are somewhere in the middle. It seems like yours is closer to the second case, that is, you need to know that the messages were processed. Message queue services have a notion of acknowledgement for that. A consumer grabs a message from a queue, and does something with it. Once it processed the message, it contacts the message queue service to acknowledge that the processing is now done. Or it may acknowledge that it failed to process it, or not acknowledge anything: in those cases, the message queue service will requeue the message. Depending on the configuration, either it will requeue it to the original queue, or use another one, such as a dead letter. This allows you to either deal with such messages automatically, or to track them manually to see what got wrong.

Always remember, that with message queue services, the message is never guaranteed to be processed once: it's either processed at least once, or once or not at all. Depending on that, design your system to handle duplicates (for instance by performing idempotent actions), or handle the cases where the message was lost.

maybe because my Service-A haven't publish the message in the bus

This case can be handled through logging, as advised by davidbak in a comment above. Note that it's not necessarily the publisher which should handle logging—it may also be the message queue service itself which logs all the messages it receives. In some cases (for instance when auditing is important), you may need to log both at the publisher side and the message queue service.

0

I would instead publish as part of the transaction that creates the event you are publishing. Therefore, if the event doesn't publish, the transaction will fail and save in the transaction log. So long as I understand you correctly, this should save the same purpose, but with the resources that you already have available. It is a good idea to publish from within the transaction anyway. This way your events are always in sync with the event logs. Otherwise, blocking will be the least of your problems...

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.