Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [rabbitmq]

The tag has no usage guidance.

2
votes
0answers
42 views

Data processing pipeline design for processing data

I have a use case for which I need to build a data processing pipeline Customer contact leads data coming from different data sources like csv, data base, api has to be first mapped to a universal ...
0
votes
1answer
70 views

RabbitMQ: how to achieve scalable one-to-one communication?

I'm trying to use RabbitMQ for two-way one-to-one communication between clients and servers. So, Client 1 should be able to communicate only with Server 1, and vice versa: The servers stay always ...
2
votes
1answer
40 views

MQ Architecture: How to handle old messages in the queue and releasing a consumer/producer upgrade

So for example if I have a producer that sends a message {id: 1} to a consumer that expects that json object. And there's a ton of messages sitting in the queue waiting to be processed. I'm also ...
-1
votes
1answer
63 views

Writing a distributed chat logger

I'm trying to write a very high scale IRC logger that can handle like 1k messages per second. My Database of choice is cassandra and maybe later I might look into scylladb. My Question is how do I ...
0
votes
1answer
125 views

Message Broker vs. iPaaS

Does anyone have suggestions/recommendations/guidance they can offer on when to use a message broker like RabbitMQ/Kafka vs. an iPaaS solution like Mulesoft/Jitterbit/Dell Boomi/Workato? Can they ...
5
votes
2answers
307 views

Fair distributed task scheduling with RabbitMQ

Problem: A single client of our system can completely flood all available resources with a massive workload. You can consider we have only one queue, and anyone can schedule any amount of work in it. ...
1
vote
2answers
229 views

Pull important messages from DLQ queue and save them in a relational database, to be analyzed and sent back later. Is it a good idea?

My project has a integration with a external system. We need to send some important informations to this system. For this, we create a micro service to connect to this external system. This micro ...
0
votes
1answer
187 views

asp.net Core 2 MVC - best way to send 1000s of emails every day

We need to send 1000s (under 10,000) per day for a few years. The data is in a SQL Server database - we have many asp.net MVC 5 and .net CORE 2 MVC sites on a Windows 2012 R2 server on IIS. The ...
1
vote
1answer
45 views

How to architect master/slave store/messages and de-duplication?

To clarify, as I'm not entirely sure how to summarise in the title. I have a redis cluster and a redis-backed scheduler (similar to sidekiq/resque, etc). My redis cluster has a master and a couple ...
2
votes
0answers
184 views

RabbitMQ: Topics and Persistence/Durability

Right now I'm using Direct Exchanges (i.e. one queue per "topic", pretty much pushing messages directly to a queue). This works OK, but I'd like the flexibility of working with Topic Exchanges, ...
1
vote
2answers
447 views

Architecture for Queuing based technology in a Micro-Service, Multi-Tenant Environment

Hopefully the title is not painfully obscure, but as it indicates, I'm really looking for advice on my architecture proposal, as the current architecture is subject to concurrency issues, performance ...
2
votes
1answer
174 views

What is the better place to create the Binding and Queue on RPC communication: Consumer or Producer?

I was talking with part of my team about the responsibility for who own the Binding and Queue creation. We are using Spring framework. In async communication they agree that the responsability to ...
8
votes
1answer
1k views

Making API calls with celery

I'm designing a system for a client where the requirements are: they upload a JSON file (one object/line) make a call to an API with the JSON object as the payload record the state (success/failure) ...