Last call to make your voice heard! Our 2022 Developer Survey closes in less than a week. Take survey.
26 votes

Message Queue. Database vs Dedicated MQ

Everything I have read on the internet suggests that using a database for a Message Queue isn't a scalable solution. The reality frequently ignored by the "don't use X because it doesn't scale" (link ...
user avatar
  • 19.9k
17 votes

Does Akka obsolesce JMS/AMQP message brokers?

Actor Model The actor model is computer science strategy for building applications that handle lots of concurrent computation and stateful processing. It's not the only strategy but it's a very well ...
user avatar
17 votes
Accepted

Using flat files vs database/API as a transport between a frontend and backend

Switching to a solution involving databases or the queuing systems mentioned by Ewan would create dependency on a new, complex system in both backend and frontend introduce unnecessary complexity and ...
user avatar
13 votes
Accepted

UML: Queue processor in a sequence diagram

As the queue is an important component in the sequence you are presenting, it should most definitely be present with a lifeline. As the consumer explicitly listens for events from the queue, I would ...
user avatar
11 votes

UML: Queue processor in a sequence diagram

Implementing Bart's answer in the wonderful PlantUML could look like this, (ab)using an entity to distinguish the queue from the participating threads: The corresponding source code: participant ...
user avatar
11 votes

Using flat files vs database/API as a transport between a frontend and backend

I don't think either solution is inheritly a bad practice, so answering which is the best practice may be difficult. I don't believe the YAGNI principal applies here if you're dealing with scale. "...
user avatar
11 votes

Message Queue. Database vs Dedicated MQ

Message Queues really come into their own when you have many of them and route messages between them, fanout to more than one consumer etc. If you just have a single 'job queue' of stuff you want to ...
user avatar
  • 64.1k
8 votes

Implementing a caching microservice by avoiding potential bottlenecks

My suggestion is that instead of worrying about whether to use an external or internal cache, your first concern should be that your booking-service does not care whether or not your are using an ...
user avatar
8 votes

Traditional Message Brokers and Streaming Data

Kafka/Kinesis is modelled as a stream. A stream has different properties than messages. Streams have context to them. They have order. You can apply window functions on streams. Although each item in ...
user avatar
  • 9,007
8 votes
Accepted

Traditional Message Brokers and Streaming Data

Kafka deals in ordered logs of atomic messages. You can view it sort of like the pub/sub mode of message brokers, but with strict ordering and the ability to replay or seek around the stream of ...
user avatar
7 votes
Accepted

How to implement a message queue over Redis?

If you want to use Redis for a message queue in Node.js and you don't mind using a module for that then you may try RSMQ - the Redis Simple Message Queue for Node. It was not available at the time ...
user avatar
  • 7,748
7 votes
Accepted

AMQP routing-keys naming anti-patterns when using topics

It seems like a mistake to me. The point of publishing a message is that you are decoupled from the component or components receiving the message. The business logic about whether the message is then ...
user avatar
  • 64.1k
7 votes

Message Queue. Database vs Dedicated MQ

As other have mentioned scale is probably not important here. The problem with using two different storage mechanisms is transactional integritety. If going with a dedicated message queue you need to ...
user avatar
7 votes

Message Bus v Mediator pattern v In Memory Bus

It's not that simple, and of course "it depends": I can't remember where, but I think I remember reading Roger Johansson write something along the lines of "Don't queue messages using ...
user avatar
  • 171
7 votes

Dealing with data arriving at a different times

Don't view the result as the outcome of a request, but as the followup of an event. The question doesn't specify much so I can only speak in abstract terms. I believe the following to be the best ...
user avatar
  • 161
5 votes
Accepted

Is having a bi directional message queue a design smell?

Communications do not usually only go one way. Parties usually communicate using request-response pairs, which are clearly not one way. I presume that you already know this, so what you are probably ...
user avatar
  • 31.4k
5 votes
Accepted

Message queue vs database for delayed tasks

Systems such as RabbitMQ (or in your case maybe look into Kafka) can offer persistence, or guaranteed delivery, as well as configurable TTL on messages. However, they are not designed as a long-term ...
user avatar
  • 319
5 votes

Using flat files vs database/API as a transport between a frontend and backend

While the good 'ol save a file and copy it to a done dir is a staple of many communication layers esp. with older main frame systems and the like. The 'anti' guys do have a point; in that it has many ...
user avatar
  • 64.1k
5 votes

Solve Synchronization

As per the wikipedia page you linked to regarding message queues: Message queues provide an asynchronous communications protocol, meaning that the sender and receiver of the message do not need ...
user avatar
  • 9,637
5 votes

Message Queue. Database vs Dedicated MQ

For practicality, main reasons for me to use a message queue are: I didn't have to reinvent the wheel. Designing even the simplest message queue using database requires at least a couple of hours of ...
user avatar
  • 3,418
5 votes
Accepted

How to deal with failing messages in DDD?

One way to deal with this is to use pull-based listeners instead of push-based. Each listener keeps track of its last read message and can request "messages since X". You could use either polling or a ...
user avatar
5 votes
Accepted

Distributed message queue, propagating queue leader/follower information

The problem with the last option is that I would still need somewhere to store queue metadata information like creation date. Other than you may store other fields (like timestamp) in Metadata ...
user avatar
  • 3,052
5 votes

How can message queues improve scalability?

It's not that queues are more scalable, its the fact that two services communicating through queue means the communication is asynchronous. Asynchronous communication is far more scalable than ...
user avatar
4 votes
Accepted

Passing notifications between programs written in different programming languages

We faced a similar problem, and used RabbitMQ for message transport, and formatted the messages using Google Protocol Buffers. RabbitMQ is excellent, it just works. Protocol buffers format the data ...
user avatar
4 votes
Accepted

What is an appropriate pattern for handling a queue of network events in C++?

While processing the messages in the queue, you probably want to call a processing function that is specific to the most-derived type of the message you are looking at. The one place where that ...
user avatar
4 votes

Message persistence in Message Queues

Consider a client that wishes to push a message to a queue. The queue manager would typically send a message acknowledgement to the client to signify that the message was successfully delivered to ...
user avatar
  • 26.3k
4 votes
Accepted

Best practice for delaying jobs

Your two main concerns here in high level design are essentially, the messaging from one or more software components to other software components, and flexible scheduling of jobs based on the age of a ...
user avatar
  • 26.3k
4 votes

How to implement "viewed: N times" functionality for an article?

I would suggest updating the counter on each view. As @amon suggested - that hard part is determining WHEN to update the counter. An old adage - no premature optimization before its time. ...
user avatar
  • 141
4 votes
Accepted

Should we be using pub/sub in our messaging stack?

I have written a similar system about a month ago, and just like you I've discovered that this is best solved by a reliable queue technology with a redrive. While some might argue using SQL as your ...
user avatar
  • 364

Only top scored, non community-wiki answers of a minimum length are eligible