Questions tagged [rabbitmq]
The rabbitmq tag has no usage guidance.
58
questions
0
votes
1
answer
70
views
Is RabbitMQ suitable for allowing SaaS customers to subscribe to events?
I work on a SaaS system which is generally a single-tenant environment. Customers (whom the SaaS instances are for) would like a way to subscribe to events, such as updates of certain data entities. ...
- 101
0
votes
1
answer
85
views
RabbitMq - Should I dynamically declare queues and make one consumer listen to all queues?
I have application X that can create "topics" where users can do operations, I want each "topic" to be a new "queue", so every "topic" operations are ...
- 83
0
votes
2
answers
133
views
Message broker design pattern best practice [closed]
I'm designing the architecture of a platform to introduce a message broker in an existing data collection web application.
This web application is currently used to upload data from excel files that ...
- 109
0
votes
0
answers
41
views
How to handle maintenance to your queuing server (e.g. rabbitMQ)
I'm new to queuing and looking into options that could be done operationally. I'm sorry if I have any conceptual misunderstandings related to queuing.
I watched this video where the presenter made a ...
- 301
-1
votes
1
answer
200
views
Message queuing systems: sequential processing with dependencies between queues
I am trying to understand Rabbitmq and how I can accomplish sequential message processing in certain circumstances only. For example if I have 2 queues. One for pricing an order and one for sending ...
- 107
0
votes
1
answer
63
views
Long-running jobs in an event-driven environment with constrained max-execution-duration
Hello we have an async event-driven system (kotlin, spring cloud stream, rabbitmq) where there might be an event FooPayloadArrived, published by an ingress rest-controller.
Processing this ...
- 115
-1
votes
2
answers
89
views
Applications interested to the same event and concurrency
I have an event pubblished in a bus(rabbit) and two application that are subscribed to this Event.
public class CreationEvent
{
public int CreationEventID {get; set; }
... other properties
}
...
- 277
5
votes
1
answer
6k
views
REST APIs vs Message brokers for Backend to Backend communication
We are a wholesale marketplace and we have three main products in our startup. The products are:
1- the marketplace app that allows customers to view products and purchase online.
2- the seller app ...
- 61
0
votes
1
answer
207
views
What limits throughput for a single topic in Kafka, Pulsar, RabbitMQ?
I am trying to assess which of Apache Kafka, RabbitMQ, and Apache Pulsar will meet my throughput requirements for a system with multiple producers and multiple consumers for a single topic.
How can ...
- 119
0
votes
1
answer
311
views
Are RabbitMQ Streams overkill for 1 machine event-driven architecture aplication?
If developing event-driven application(more precisely event sourcing application) that will run solely on 1 machine are RabbitMQ Streams an overkill, or maybe a misfit?
In theory RabbitMQ Streams seem ...
- 253
2
votes
3
answers
2k
views
RabbitMQ or REST API Public
Currently working on designing an application that will take in JSON data from remote client applications. The data these client applications are sending is going to go into a Message Broker, namely ...
- 21
0
votes
2
answers
100
views
Do I need a Message processing service before RabbitMQ (or any other message broker)
I hope that's a proper place to ask my question.
I am wondering how is best to integrate RabbitMq (or any other message broker) into my project.
There are 2 options:
Simply collect all messages, ...
- 103
1
vote
1
answer
2k
views
Rabbitmq create queues dynamically based on number of users
I have a scenario where I have to route a list of messages that I get to respective users.
For suppose if I have
messages = [
{ text: 'hi', user_id: 1 },
{ text: 'hi', user_id: 2 },
{ text: 'hi',...
- 119
2
votes
2
answers
1k
views
How to share API between microservices?
Currently, our architecture uses an "API-first" approach in building our product. This product is divided across multiple teams handling different microservices.
The above image succinctly ...
- 165
0
votes
0
answers
508
views
High availability RabbitMQ using gRPC as transport layer?
I have an idea to use gRPC for all microservices communication for the sake of simplicity.
Now there is a service that needs high availability, So the idea is to use RabitMQ as a message receiver from ...
- 687
2
votes
1
answer
382
views
How to decouple messaging middleware from microservices?
We have a bunch of microservices written in different programming languages. These all communicate with each other over RabbitMQ(AMQP) middleware. There are consumers and publishers in each ...
- 165
0
votes
1
answer
129
views
Direct communication between message-driven bounded context
My project comprises of serveral bounded context which communicate with each other with RabbitMQ
The front end of the project is written in C# and the backend is in Java.
The RabbitMQ communication ...
- 165
0
votes
0
answers
111
views
Redundancy approach for Pub-Sub application with events replication
I'm new to the microservices field.
I'm looking for the right approaches regarding how to deal with the challenge of redundancy of a pus-sub application.
Assume my application has has the below ...
- 101
0
votes
1
answer
53
views
How to dispatch sets of items to the relevant (LB) exclusive consumers with RabbitMQ?
I am trying to process sets of items (also, different sets have different items) in specific RabbitMQ consumers (one for each set of items) that would be created on-demand and are dispatched to the ...
- 319
-1
votes
1
answer
79
views
How to write package for two frameworks
I'm developing a package for Laravel & Lumen in the same code base, but I encountered some problems.
Well, in the beginning, I thought it could have the same code and work for both frameworks, ...
- 17
1
vote
0
answers
301
views
RabbitMQ messages reliability for at-least-once delivery
I wanted to ask what are the possibilities of maintaining reliability when exchanging messages between microservices, when one of those message is rejected. As of today, we don't want to lose messages,...
- 77
-1
votes
2
answers
732
views
Common library for classes and communication between microservices?
I have been working on a project that is being built around a microservices architecture. We are thinking on using a message broker, such as RabbitMQ, for both synchronous (via RPC) and asynchronous ...
- 245
2
votes
1
answer
520
views
AMQP messaging: How to generate documentation for consumers?
My application publishes various messages (e.g., telemetry, changed-events, ...) to a RabbitMQ message-broker. The users have no access to the source-code and therefore, will need a separate ...
0
votes
0
answers
360
views
Integrating HTTP / Webhooks with Message Queues
I'm working at a project which integrates several Applications mostly SaaS Applications. The SaaS solutions have all the possibilities to hook into the internal event system with webhooks. The ...
- 111
0
votes
0
answers
102
views
Is queue is the right choice for server request retries
I have server A that received millions of requests per day and for every request it notifies server B about it.
sometimes server B is down (maintenance, bugs etc..) so requests from server A got lost.
...
- 109
1
vote
1
answer
137
views
Best approach to update the User Interface on consumed event(s)?
I am developing a microservice architecture where the API publishes a message through RabbitMQ. One or more consumers will read the message, preform its functions then if successful acknowledge the ...
- 121
-2
votes
2
answers
120
views
Inter-microservice communication with gRPC without hard-coding service endpoints
I'm building a microservices architecture that requires microservices to communicate with each other. Searching Google has led me to believe that gRPC is the preferred method for achieving this. ...
- 107
1
vote
0
answers
613
views
Approach to setup multi-tenant RabbitMQ workers in Python
I'm migrating my current application to multi-tenant setup.
Now I've multiple rabbitmq workers to process async job, publish and consume integration events, and other stuffs. I'm planning to use ...
- 167
-2
votes
1
answer
47
views
Synchronisation of different "channel" in an asynchronous way
I have a scenario where a service needs to update back and forth information from different channels (and each channel should have the same information, e.g inventory of products).
It all works fine ...
- 107
0
votes
1
answer
88
views
Process AMPQ messages both reliably and fast
I am new to RabbitMQ and herewith I want to make sure that I am not missing out on some advanced RabbitMQ feature or pattern I am not aware of.
I need to develop a reliable system that processes a ...
- 139
0
votes
2
answers
495
views
Moving single tenant application with queue to multi tenants web application
I need to move a single-tenant web application to a multi-tenant (about 100 tenants) web application. Tenants are going to share the same application but each tenant is going to have its own database (...
- 121
1
vote
3
answers
13k
views
Rabbitmq and multiple subscribers
Is it possible for a queue to have multiple consumers?
I am working on a system where a single queue needs to be accessed by multiple consumers for different micro-services. Is it possible for the ...
1
vote
2
answers
770
views
Implementing RESTful API in front of Event based microservices
I'm working on a system that implements multiple microservices which communicate via a RabbitMQ messaging bus.
These microservices are created using python with the pika library (to publish messages ...
- 19
0
votes
2
answers
1k
views
How to send queue messages to one queue in rabbitmq?
I want to use rabbitmq to share domains events between microservices. Due to the fact that queues are round-robin, I need to define one queue per event source and event consumer.
For example, cars-ms ...
1
vote
0
answers
28
views
Consumer triggered entity data agregation
I am currently using RabbitMQ to aggregate the output of various data sources.
Functionally the process is the following:
The user enters a product name.
The request is sent to the different sources ...
- 119
1
vote
1
answer
195
views
How do I negotiate messages to avoid errors resulting from duplicate messages?
My system looks the following:
I have N message producers, N RabbitMQ instances, N message consumers and 1 service. Producers produce the same messages while consumers consume the same messages. Here ...
- 111
2
votes
0
answers
106
views
Distributing tasks to several clients
I describe a very very simple scenario. I try to develop a system where many machines can help another one (server). I tried RabbitMQ and Redis, but both don't fulfil all requirements.
I have 1-...
- 121
0
votes
1
answer
729
views
Data replication with microservices using a messaging broker
I have a microservice based application like so:
User.Microservice - stores , user information.
Product.Microservice - stores products that user created.
Order.Microservices - stores product orders
...
2
votes
0
answers
321
views
Data pipeline architecture: airflow triggered by message broker
Let us say we have:
a web app with a Postgres DB that produces data over time,
another DB optimized for analytics that we would like to populate over time.
My goal is to build and monitor an ETL ...
- 151
0
votes
1
answer
739
views
What's the use of event sourcing and asynchronous message queues for sending commands
In short: Out of order commands end up with out of order events, so does the ordering of the events matter in such situations?
Since we are uncertain about the order of commands when an asynchronous ...
- 209
5
votes
1
answer
205
views
MQ integration: How to notify consumers of upcoming message format changes?
We have multiple microservices communicating over MQ. As MQ messages are the interface/contract between the services, whenever we make changes to the MQ message published by a service we need to make ...
- 151
0
votes
2
answers
1k
views
Best design for a AMQP work queue
I have a system where a user uploads a file to import some users but it's necessary to do some validations and this take some time. I would like to create a work queue with RabbitMQ and the user will ...
- 1
2
votes
2
answers
807
views
RabbitMQ Consumer Architecture
I have been investigating using RabbitMQ for a publish/subscribe pattern across micro-services where aysnc calls are appropriate but I am having trouble understanding some concepts and could not find ...
- 269
1
vote
1
answer
124
views
Queue based indexing & multiple languages
I'm thinking about refactoring my content indexing process to queues. Right know updates happen directly when editors change content. It is difficult to do maintance works on the search engine too ...
- 207
0
votes
0
answers
41
views
AMQP integration
I’m trying to make a AMQP integration layer for my app. This layer would only consume messages from my customer’s queues and the result would be a specific action in my application.
My application is ...
3
votes
1
answer
305
views
Data processing pipeline design for processing data
I have a use case for which I need to build a data processing pipeline
Customer contact leads data coming from different data sources like csv, data base, api has to be first mapped to a universal ...
- 139
1
vote
1
answer
1k
views
RabbitMQ: how to achieve scalable one-to-one communication?
I'm trying to use RabbitMQ for two-way one-to-one communication between clients and servers. So, Client 1 should be able to communicate only with Server 1, and vice versa:
The servers stay always ...
- 161
2
votes
1
answer
415
views
MQ Architecture: How to handle old messages in the queue and releasing a consumer/producer upgrade
So for example if I have a producer that sends a message
{id: 1}
to a consumer that expects that json object. And there's a ton of messages sitting in the queue waiting to be processed. I'm also ...
- 123
-1
votes
1
answer
69
views
Writing a distributed chat logger
I'm trying to write a very high scale IRC logger that can handle like 1k messages per second. My Database of choice is cassandra and maybe later I might look into scylladb.
My Question is how do I ...
- 101
2
votes
1
answer
945
views
Message Broker vs. iPaaS
Does anyone have suggestions/recommendations/guidance they can offer on when to use a message broker like RabbitMQ/Kafka vs. an iPaaS solution like Mulesoft/Jitterbit/Dell Boomi/Workato? Can they ...
- 397