Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [rabbitmq]

The tag has no usage guidance.

0
votes
0answers
24 views

AMQP integration

I’m trying to make a AMQP integration layer for my app. This layer would only consume messages from my customer’s queues and the result would be a specific action in my application. My application is ...
0
votes
0answers
13 views

SOAP message exchange issue

We have a system that receive payment information for invoices from the bank system. It works like that: Invoice is created in our system Payer pays for the invoice from system of Bank Bank connects ...
2
votes
0answers
59 views

Data processing pipeline design for processing data

I have a use case for which I need to build a data processing pipeline Customer contact leads data coming from different data sources like csv, data base, api has to be first mapped to a universal ...
0
votes
1answer
96 views

RabbitMQ: how to achieve scalable one-to-one communication?

I'm trying to use RabbitMQ for two-way one-to-one communication between clients and servers. So, Client 1 should be able to communicate only with Server 1, and vice versa: The servers stay always ...
2
votes
1answer
47 views

MQ Architecture: How to handle old messages in the queue and releasing a consumer/producer upgrade

So for example if I have a producer that sends a message {id: 1} to a consumer that expects that json object. And there's a ton of messages sitting in the queue waiting to be processed. I'm also ...
-1
votes
1answer
64 views

Writing a distributed chat logger

I'm trying to write a very high scale IRC logger that can handle like 1k messages per second. My Database of choice is cassandra and maybe later I might look into scylladb. My Question is how do I ...
0
votes
1answer
158 views

Message Broker vs. iPaaS

Does anyone have suggestions/recommendations/guidance they can offer on when to use a message broker like RabbitMQ/Kafka vs. an iPaaS solution like Mulesoft/Jitterbit/Dell Boomi/Workato? Can they ...
6
votes
2answers
381 views

Fair distributed task scheduling with RabbitMQ

Problem: A single client of our system can completely flood all available resources with a massive workload. You can consider we have only one queue, and anyone can schedule any amount of work in it. ...
1
vote
2answers
365 views

Pull important messages from DLQ queue and save them in a relational database, to be analyzed and sent back later. Is it a good idea?

My project has a integration with a external system. We need to send some important informations to this system. For this, we create a micro service to connect to this external system. This micro ...
0
votes
1answer
226 views

asp.net Core 2 MVC - best way to send 1000s of emails every day

We need to send 1000s (under 10,000) per day for a few years. The data is in a SQL Server database - we have many asp.net MVC 5 and .net CORE 2 MVC sites on a Windows 2012 R2 server on IIS. The ...
1
vote
1answer
62 views

How to architect master/slave store/messages and de-duplication?

To clarify, as I'm not entirely sure how to summarise in the title. I have a redis cluster and a redis-backed scheduler (similar to sidekiq/resque, etc). My redis cluster has a master and a couple ...
2
votes
0answers
209 views

RabbitMQ: Topics and Persistence/Durability

Right now I'm using Direct Exchanges (i.e. one queue per "topic", pretty much pushing messages directly to a queue). This works OK, but I'd like the flexibility of working with Topic Exchanges, ...
1
vote
2answers
525 views

Architecture for Queuing based technology in a Micro-Service, Multi-Tenant Environment

Hopefully the title is not painfully obscure, but as it indicates, I'm really looking for advice on my architecture proposal, as the current architecture is subject to concurrency issues, performance ...
2
votes
1answer
179 views

What is the better place to create the Binding and Queue on RPC communication: Consumer or Producer?

I was talking with part of my team about the responsibility for who own the Binding and Queue creation. We are using Spring framework. In async communication they agree that the responsability to ...
8
votes
1answer
2k views

Making API calls with celery

I'm designing a system for a client where the requirements are: they upload a JSON file (one object/line) make a call to an API with the JSON object as the payload record the state (success/failure) ...