0

If developing event-driven application(more precisely event sourcing application) that will run solely on 1 machine are RabbitMQ Streams an overkill, or maybe a misfit?

In theory RabbitMQ Streams seem nicer than manual callback in code that are hard to track and monitor, but then on the other hand RabbitMQ Streams seem designed for microservices that run on multiple machines with many replicas...

note: I am mentionining RabbitMQ streams as an example, I presume similar question can be asked about Redis Streams.

1
  • 1
    A persistent queue can be a the answer if you must never lose an event. If that is not important, it is probably better to avoid the complexity it brings. Commented Aug 4, 2021 at 16:36

1 Answer 1

2

If you need to use multiple processes, a message queue can be appropriate. If you only have one process one of the main advantages will be lost, but that does not mean it cannot provide some utility. The question becomes if the utility is worth the extra complexity. I would also suggest that a message queue are not a direct replacement for callbacks or events since they are quite a bit more heavy weight.

Perhaps looking at the suggested use cases will reveal more

large fan-outs: where many applications need to read the same messages (with traditional queues, that would require declaring a queue per application and delivering a copy of the same message to each of them)

Probably not relevant if you are only on one machine.

large backlogs: streams store messages on disk, not in-memory, so the only limit is the disk capacity replay & time-traveling: consumers can attach anywhere in a stream, using an absolute offset or a timestamp, and they can read and re-read the same data

These seem like a valid use case if you need persistence.

high throughput: streams are super fast compared to traditional queues, several orders of magnitude faster

This is probably compared to older inter-process queues. If you only have one process you could just use a regular queue, or a stack if you do not intend to remove messages.

To conclude, if you have multiple processes or persisting all messages to disk seem appealing, RabbitMQ streams might be worth a try. If not, I fail see what advantage it would bring.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.