Questions tagged [pubsub]

Publish/subscribe is a method for wiring software components together. One component publishes data, and subscribers receive events containing the data. This decouples the components so they do not need to know about each other; and it allows for configuration to wire components together without recompiling code.

Filter by
Sorted by
Tagged with
1 vote
1 answer
122 views

Event bus vs PubSub

I have been using queues and PubSub patterns for years but never really came across the terminology "event bus". After some googling I didn't really find a clear distinction between "...
user avatar
  • 119
0 votes
0 answers
63 views

Bi-directional messaging using Zero MQ pub sub pattern

I am trying to design a system that can be safely used to share messages between processes on the same system using ZeroMQ. I have chosen to use a central broker that binds to a particular port. All ...
user avatar
0 votes
0 answers
53 views

Design : How to divide the responsibility in micro services

I have two micro services built using spring boot, One micro service is: "payment" Which handles actions related to payments. Like creating a payment link. Getting the latest status of ...
user avatar
  • 101
1 vote
2 answers
97 views

Make infinite loop handleable via delay?

I have a Node.js micro-service architecture-based back-end. Services communicate with each other via PubSub. E.g. upon a request to service A, service A messages either service B or service C via ...
user avatar
  • 119
0 votes
0 answers
14 views

When to create a new socket in ZMQ or add a new topic to an old one in pub/sub?

I have an application that remotely speaks to another application. There's a lot of coordination between the two applications, but they can exist on their own with out the other, just with restricted ...
user avatar
  • 948
0 votes
1 answer
45 views

What kind of bottlenecks are possible when implementing a distributed workflow using a central point of control?

I was going through https://docs.microsoft.com/en-us/azure/architecture/patterns/choreography, the document mentions in the drawbacks of having a central orchestrator that it can become a performance ...
user avatar
  • 145
0 votes
0 answers
187 views

gRPC client - to - client communication between other Lan network

I have a question about gRPC communication. I know gRPC has 4 models communication: unay, server-streaming, client-streaming and bi-direction. And the terms "client" and "server" ...
user avatar
0 votes
0 answers
61 views

Redundancy approach for Pub-Sub application with events replication

I'm new to the microservices field. I'm looking for the right approaches regarding how to deal with the challenge of redundancy of a pus-sub application. Assume my application has has the below ...
user avatar
  • 101
1 vote
0 answers
256 views

Broker queue and topic naming conventions

I'm in the process of adding a message broker (ActiveMQ Artemis) to a system currently and while I have my own notions on how I can divide up the namespace, it seems like a good time to get opinions ...
user avatar
0 votes
1 answer
66 views

Pubsub model - publisher broadcast to remote subscribers

With a single publisher(go-routine) and multiple subscribers(go-routine) on same machine, below message hub help create pubsub model: type PublisherHub struct { subscribers map[*subscribmediator....
user avatar
  • 2,225
0 votes
0 answers
37 views

How to narrow time window for published data

I have a system organized in microservices: we can simplify the structure with backend server, frontend server and client. Data are quite limited (hundreds of redis keys) and each client can recevie ...
user avatar
0 votes
5 answers
263 views

How to deail with a hypothetical situation in which a pub/sub cycle gets into an unending recursive loop?

Let me explain what I mean. Imagine A subs to event b. In such case A pubs event a. B subs to event a. In this case B subs b. This is a full-blown circle. How does a pub/sub cycle deal with such ...
user avatar
  • 27
3 votes
1 answer
150 views

The publish-subscribe pattern: garbage-collecting old subscriptions

I've been studying distributed systems design, in particular the Udi Dahan's class. He talks about the publish-subscribe pattern as a common pattern in messaging-oriented designs. There's obviously a ...
user avatar
  • 715
-4 votes
1 answer
63 views

application can not handle certain amount of RPS when 3k+ sessions connected

i have a problem in my scalable Pub/Sub application that it can not take more than 30 RPS when big amount of sessions are open against it. first i will explain the application structure. the ...
user avatar
  • 21
1 vote
2 answers
78 views

How to agree on message schema in a Publish–subscribe pattern

I'm working on a project that uses PubSub(GCP), my question is not specific to GCP, it's more regarding to the architectural pattern(I'm used to statically typed languages, and I have a hard time ...
user avatar
  • 21
2 votes
1 answer
141 views

Publisher-Subscriber architecture with central registry

Problem description Publisher-Subscriber architecture with a central registry where agents can either promote their capabilities or search for a given capability. The project must be developed with ...
user avatar
  • 129
-1 votes
1 answer
126 views

pub/sub events to/from microservices in different languages

I'm starting a brand new project implementing microservices with domain driven design. We will have microservices written in different languages like C#, Python, and Node. I'm thinking about hosting ...
user avatar
1 vote
0 answers
195 views

Flowchart symbol or pattern for Publisher/Subscriber or Listen/Notify

We have a system consisting of Detectors Converters/Transformers Business Rules Engines and State Managers Views/Displays/Applications There are several potential ways these may interact and share ...
user avatar
2 votes
0 answers
47 views

Data change event publishing by microservice vs separate application - best practice

Suppose there is a microservice that has an restful HTTP API for CRUD operations on a database - nothing fancy, but there is a journal table of all changes recorded for audit purposes. Suppose also ...
user avatar
  • 1,836
1 vote
0 answers
180 views

Event-based communication between microservices

I'm currently looking into NestJS to use as the framework for a new e-commerce project. The goal is to build multiple loosely-coupled services, which can communicate with each other using the pub/sub ...
user avatar
  • 111
0 votes
0 answers
74 views

Preferrable way to send push messeges?

I am making a chat-app using the XMPP protocol and ejabberd as XMPP server. I want to build a push notification functionality for the app. The app is similar to WhatsApp. I came to know three ...
user avatar
2 votes
2 answers
341 views

Pros and cons of consuming events in batches

Suppose I have a message broker that allows consumers to specify a batch size. What are the advantages and disadvantages of consuming events in batches compared to consume them one by one? The ...
user avatar
  • 249
4 votes
1 answer
221 views

Pub-Sub pattern and instance replication

I am porting an event-driven architecture to a container environment (Docker) and I wonder how to take advantage of replication while using pub-sub: if I just increment replicas I end up with many ...
user avatar
-1 votes
2 answers
108 views

Using Apache Kafka for rare events [closed]

There are multiple microservices which are responsible for storing data dependent from user, e.g. user preferences on some products, user orders, etc. There is a scenario when user is deleted from ...
user avatar
  • 111
3 votes
1 answer
111 views

Run a job every hour with Lambda or something else?

I am working on a SaaS project that will have a trial when the trial is ending I get a webhook notification when 3 days are remaining. I do some stuff with this and one of the things is I update user....
user avatar
  • 161
0 votes
1 answer
457 views

How can an event subscriber catch up with missed events?

At a conceptual level, what ways can a subscriber catch up with events it has potentially missed but needs to know about at a later time. For example, a service is running and subscribes to events on ...
user avatar
1 vote
1 answer
69 views

What is a good strategy for managing PubSub subscriptions

I am working with Reddis but this could apply to any PubSub implementation. Essentially we are trying to implement an event system using Reddis's PubSub capabilities. As an example UserA creates a ...
user avatar
0 votes
1 answer
54 views

What types of Google Cloud application should I consider for something running as a continually available service that subscribes to a pub/sub?

I want to create an application that will run as a service that subscribes to a Google Cloud pub/sub topic, and delivers emails based on 'alerts' that it receives through the pub/sub subscription. It ...
user avatar
0 votes
1 answer
203 views

Which message pattern should I use?

I'd like to develop a microservice architecture application that sends text messages from one communication provider to another and I'm just considering changing architecture would be best suited to ...
user avatar
  • 11
1 vote
1 answer
142 views

Event sourcing access by reference

In an event sourcing architecture, what is the typical pattern for passing information about related objects (aggregates)? For example, in a order processing system, should OrderCreated event (...
user avatar
  • 139
0 votes
2 answers
3k views

Maintaining order of events with multiple consumers in a typical pub sub scenario

I am using Kafka. I am developing a simple e-commerce solution. I have a non-scalable catalog admin portal where products, categories, attributes, variants of products, channels, etc are updated. For ...
user avatar
1 vote
1 answer
141 views

Handling application-wide notifications from various layers of the application?

I have recently started updating portions of our corporate platform, and I've been trying to find ways to make the entire platform more efficient, better structured, and just cleaner in general. With ...
user avatar
  • 1,317
0 votes
1 answer
2k views

Processing only once the same message produced by two producers

If I have two different producers that could produce the same message for a Kafka broker, how can I ensure that only one of the two message occurrences gets processed? Is the only way to have an ...
user avatar
2 votes
2 answers
179 views

Name of this pattern

I create an object that holds the status of a job. The job has many stages, some concurrent. Worker threads constantly monitor the status object to see if it's ready for the next stage to begin, if it ...
user avatar
  • 198
0 votes
1 answer
2k views

How to load test Publisher/Subscribing queue?

I am trying to load test a queue that works on a pub sub pattern and can't figure out how response time is to be calculate and the TPS(transactions per second). Problem statement : The pub sub ...
user avatar
2 votes
1 answer
348 views

Architecture pattern for temporary message queue subscriptions with websocket clients

We use Google Pub/Sub for event streams which we want consumed by transient websocket clients. What is a good pattern for creating subscriptions and cleaning them up when the client is no longer ...
user avatar
  • 29
0 votes
1 answer
312 views

Consuming messages directly from a queue which belongs to a third party system

We are working on a project which interacts with multiple third party systems and they need each other's data to function. Now, for some data which needs to be in sync in multiple systems, we are ...
user avatar
50 votes
4 answers
25k views

Ways to share DTO across microservices?

My scenario is as follows. I am designing a system designed to receive data from various types of sensors, and convert and then persist it to be used by various front-end and analytics services later....
user avatar
1 vote
1 answer
262 views

How do I set up short-lived queues?

This scenario seems pretty ordinary, and yet, strangely, messaging systems (like Google Cloud PubSub and Task Queues and ActiveMQ) do not seem to support it -- they assume that topics/queues are long-...
user avatar
  • 1,080
3 votes
1 answer
5k views

What is a good diagrammatic way to represent async event communication between two systems?

What is a good diagrammatic way to represent asynchronous event communication between two systems (the pub sub model)? I am not looking for tools but more of a representation. Is it a sequence diagram,...
user avatar
1 vote
1 answer
542 views

Publish and Subscribe (data transfer) with permanently offline nodes. Are message queues a good fit?

The general question is what kind of mechanism can I use to transfer data to and from publishers and subscribers where publishers or subscribers can be permanently offline? Can message queues be used ...
user avatar
  • 151
2 votes
3 answers
191 views

Is this an inefficient use of the Subscriber (PubSub) pattern?

I have a Publisher object that raises an event at regular intervals (think of it something like a clock tick). I then have 100s (or 1000s) of subscriber objects (all instances of a single type or two) ...
user avatar
  • 3,302
5 votes
1 answer
2k views

pubsub with multiple instances of each consumer

I'm currently looking into setting up a publish subscribe messaging infra structure for our microservice based platform. The new setup is meant to replace our current kafka based one, with something ...
user avatar
  • 161
2 votes
1 answer
452 views

Should API gateway wait until all micro services are done?

for our project, we are using micro services with API gateway pattern. One of the features that the client do is to generate a report. Client, fills a form and uploads a file and our system will ...
user avatar
  • 123
9 votes
2 answers
7k views

Relationship between RESTful URIs and PubSub topics

I am designing a system which fits quite nicely into a RESTful architecture. Users can navigate a resource hierarchy from a root node, each resource links to other resources, resources have URIs etc. ...
user avatar
  • 261
2 votes
1 answer
529 views

Is publish–subscribe pattern suitable for my case?

I'm planning to use publish–subscribe pattern with Azure Service Bus in my logistics application, but I'm confused whether publish–subscribe pattern is suitable for my case. My scenarios: Users will ...
user avatar
  • 159
2 votes
2 answers
2k views

Publish / Subscribe via HTTP Callbacks?

My team is tasked with creating a publish/subscribe system for incoming REST messages. 99% of the time, this system will be used for notifications between different processes on the same cpu, but we ...
user avatar
0 votes
0 answers
177 views

Is NSNotificationCenter an implementation of a PubSub model?

I am trying to understand the subtleties of Publisher-subscriber and similar notification mechanisms. I was checking Cocoa NSNotificationCenter, but it's unclear to me if it can be defined as a PubSub,...
user avatar
1 vote
0 answers
259 views

Lazy loading collection data through granular subscriptions and OOP (Meteor.js)

I am transforming my Meteor collection to a class I've created: People = new Mongo.Collection('people', { transform: function(doc) { return new Person(doc); } }); Person has a bunch of ...
user avatar
  • 283
5 votes
3 answers
1k views

Are imperative events an anti-pattern?

Are events which are intended to tell listeners to carry out explicit actions an anti-pattern? Imagine a worker service which raises the event WorkStarted at its beginning and the event WorkFinised ...
user avatar
  • 181