Questions tagged [rabbitmq]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
1answer
50 views

Data replication with microservices using a messaging broker

I have a microservice based application like so: User.Microservice - stores , user information. Product.Microservice - stores products that user created. Order.Microservices - stores product orders ...
1
vote
0answers
36 views

Data pipeline architecture: airflow triggered by message broker

Let us say we have: a web app with a Postgres DB that produces data over time, another DB optimized for analytics that we would like to populate over time. My goal is to build and monitor an ETL ...
0
votes
1answer
70 views

What's the use of event sourcing and asynchronous message queues for sending commands

In short: Out of order commands end up with out of order events, so does the ordering of the events matter in such situations? Since we are uncertain about the order of commands when an asynchronous ...
4
votes
0answers
81 views

MQ integration: How to notify consumers of upcoming message format changes?

We have multiple microservices communicating over MQ. As MQ messages are the interface/contract between the services, whenever we make changes to the MQ message published by a service we need to make ...
0
votes
2answers
105 views

Best design for a AMQP work queue

I have a system where a user uploads a file to import some users but it's necessary to do some validations and this take some time. I would like to create a work queue with RabbitMQ and the user will ...
0
votes
2answers
91 views

RabbitMQ Consumer Architecture

I have been investigating using RabbitMQ for a publish/subscribe pattern across micro-services where aysnc calls are appropriate but I am having trouble understanding some concepts and could not find ...
0
votes
0answers
62 views

Using rabbitMQ for an order management system

I am building an e-commerce app that allows you to buy products from an online store. I am struggling to understand if using a queue will be helpful. if so, how can I make use of it effectively? Let'...
0
votes
0answers
33 views

How many exchanges when using scatter gather and CQRS?

I am trying to learn more about CQRS and enterprise integration in my spare time. I have created an application, which uses CQRS and the Scatter Gather Pattern. There are two web APIs for the CQRS ...
1
vote
1answer
55 views

Queue based indexing & multiple languages

I'm thinking about refactoring my content indexing process to queues. Right know updates happen directly when editors change content. It is difficult to do maintance works on the search engine too ...
0
votes
0answers
37 views

AMQP integration

I’m trying to make a AMQP integration layer for my app. This layer would only consume messages from my customer’s queues and the result would be a specific action in my application. My application is ...
0
votes
0answers
16 views

SOAP message exchange issue

We have a system that receive payment information for invoices from the bank system. It works like that: Invoice is created in our system Payer pays for the invoice from system of Bank Bank connects ...
2
votes
0answers
100 views

Data processing pipeline design for processing data

I have a use case for which I need to build a data processing pipeline Customer contact leads data coming from different data sources like csv, data base, api has to be first mapped to a universal ...
0
votes
1answer
358 views

RabbitMQ: how to achieve scalable one-to-one communication?

I'm trying to use RabbitMQ for two-way one-to-one communication between clients and servers. So, Client 1 should be able to communicate only with Server 1, and vice versa: The servers stay always ...
2
votes
1answer
161 views

MQ Architecture: How to handle old messages in the queue and releasing a consumer/producer upgrade

So for example if I have a producer that sends a message {id: 1} to a consumer that expects that json object. And there's a ton of messages sitting in the queue waiting to be processed. I'm also ...
-1
votes
1answer
64 views

Writing a distributed chat logger

I'm trying to write a very high scale IRC logger that can handle like 1k messages per second. My Database of choice is cassandra and maybe later I might look into scylladb. My Question is how do I ...
1
vote
1answer
341 views

Message Broker vs. iPaaS

Does anyone have suggestions/recommendations/guidance they can offer on when to use a message broker like RabbitMQ/Kafka vs. an iPaaS solution like Mulesoft/Jitterbit/Dell Boomi/Workato? Can they ...
6
votes
2answers
680 views

Fair distributed task scheduling with RabbitMQ

Problem: A single client of our system can completely flood all available resources with a massive workload. You can consider we have only one queue, and anyone can schedule any amount of work in it. ...
1
vote
2answers
824 views

Pull important messages from DLQ queue and save them in a relational database, to be analyzed and sent back later. Is it a good idea?

My project has a integration with a external system. We need to send some important informations to this system. For this, we create a micro service to connect to this external system. This micro ...
0
votes
1answer
409 views

asp.net Core 2 MVC - best way to send 1000s of emails every day

We need to send 1000s (under 10,000) per day for a few years. The data is in a SQL Server database - we have many asp.net MVC 5 and .net CORE 2 MVC sites on a Windows 2012 R2 server on IIS. The ...
1
vote
1answer
211 views

How to architect master/slave store/messages and de-duplication?

To clarify, as I'm not entirely sure how to summarise in the title. I have a redis cluster and a redis-backed scheduler (similar to sidekiq/resque, etc). My redis cluster has a master and a couple ...
2
votes
0answers
282 views

RabbitMQ: Topics and Persistence/Durability

Right now I'm using Direct Exchanges (i.e. one queue per "topic", pretty much pushing messages directly to a queue). This works OK, but I'd like the flexibility of working with Topic Exchanges, ...
1
vote
2answers
776 views

Architecture for Queuing based technology in a Micro-Service, Multi-Tenant Environment

Hopefully the title is not painfully obscure, but as it indicates, I'm really looking for advice on my architecture proposal, as the current architecture is subject to concurrency issues, performance ...
2
votes
1answer
188 views

What is the better place to create the Binding and Queue on RPC communication: Consumer or Producer?

I was talking with part of my team about the responsibility for who own the Binding and Queue creation. We are using Spring framework. In async communication they agree that the responsability to ...
8
votes
1answer
2k views

Making API calls with celery

I'm designing a system for a client where the requirements are: they upload a JSON file (one object/line) make a call to an API with the JSON object as the payload record the state (success/failure) ...