Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

Hot answers tagged

38

My advice? Do not share these DTOs among the applications in any kind of library. Or at least don't do this right now. I know, seems very counter-intuitive. You are duplicating code, right? But this is not a business rule, so you can be more flexible. The service that sends the DTO needs to be rigid in his message contract, like an Rest API. The service ...


22

I've run into some difficulties thus far I'd like to document here. How do you handle reconnect logic? This is a hard problem and an especially hard problem in designing and implementing a message queue. Messages must be able to queue up somewhere when consumers are offline, so a simple pub-sub is not strong enough, and consumers need to reconnect in a ...


19

Yep, you are definitely missing something. Gotos would typically be used, like you said, to perform a one-way transfer of control. However, events do not do that. When the code fires the event, it knows full well that once event is published (or processed, queued, fired... etc) code execution will resume on the very next line in the code that generated ...


12

When it comes to Microservices, services' development life cycles should be independent too.* Different SLDC and different dev teams in a real MS system, there could be several teams involved in the development of the ecosystem, each of which in charge of one or more services. In turn, these teams might be located in different offices, cities, countries, ...


9

Mapping URIs (from REST resources) to topics, depending on what your are modeling, could result in a perfect mapping. Through a simple example, i will try to give you some hint about how to perform this mapping. I would take as an example, two APIS: a REST API (resources) and MQTT API (topics). By the way, MQTT is a pub/sub protocol for the Internet of ...


8

I'm trying to design every service to be as independent as possible You should be publishing events. Events are certain type of messages that represent a solid fact about something that has happened at a particular point in time. Each service should have a very well defined responsibility, and should have the responsibility of publishing the events related ...


6

The question is a bit vague (e.g. what exactly do you mean by a third party?), but to try and answer anyway: I think you're looking for the Observer pattern. The intent of the pattern is to allow entities to be notified when the state of some object changes, while decoupling the two sides. I.e., the observers don't know anything about the subject except ...


6

The way to decide what an event should represent is the good old Single Responsibility Principle. If, for instance, you have a long-running tax computation process and an incremental GUI visualization that displays its progress, then the tax processor should raise "started", "progress" or "finish" events. A GUI class should listen to these events and do ...


6

There are a couple of differences. First, when a code executes GOTO, it gives up control, and there's no guarantee that it will regain control. A publisher in pub/sub, however, will keep running and performing its logic, sending off messages as appropriate. Its behavior is understandable and predicable. Secondly, the subscriber will receive messages, and ...


5

It is none of your business what I intend to do when your API raises an event. Maybe I will show an icon, maybe I will order coffee for the lead developer, start a music video, whatever I do, it's my business not yours. The point of the event is to notify those that are interested that something has happened, after that, it's no longer your conceern. As a ...


5

I'm looking for a way to clearly diagram how multiple applications communicate via a service bus. That would be awesome. Unfortunately, I don't think it is possible for configurations of any complexity. A sequence diagram for simple scenarios is probably as good as anything else. Components on a bus both publish and subscribe to messages. Some of the ...


5

If you want to use Redis for a message queue in Node.js and you don't mind using a module for that then you may try RSMQ - the Redis Simple Message Queue for Node. It was not available at the time this question was asked but today it is a viable option. If you want to actually implement the queue yourself as you stated in your question then you may want to ...


5

Yes, you need two Worker Queues A and B https://www.rabbitmq.com/tutorials/tutorial-two-dotnet.html and one Fan Out routing which sends messages to both queues https://www.rabbitmq.com/tutorials/tutorial-three-dotnet.html So in rabbitmq you publish to a fan out Exchange and read from persistent queue A or B both of which get a copy of the message from ...


4

ActiveMQ does support this and it's a general feature of JMS. Here's a link pointing to a way of doing this. I can't find the specific documentation at the moment but my recollection is that you can set up general queue configurations using wildcards like FOO.* Then you can write to FOO.BAR and ActiveMQ will create it. It will exist as long as there ...


3

If anyone else finds this question relevant, I'm following-up with my chosen solution. Bottom line: when you can't find a simple solution to a seemingly simple problem, it may be a sign that analysis and refactoring of the existing design is needed. Analyzing my code, I realized that the Sequence class had fairly overloaded accessors that as designed could ...


3

I wouldn't worry about misusing the pattern but in the scenarios you describe I don't think you need it. PubSub seems to be the popular default at the moment for any messaging application but there is a simpler version of it for when you don't need to fan out or fan in the messages. Just use messaging. There are different forms:, http, rpc, MQ, callbacks....


3

The UML sequence diagram is well adapted to represent an interaction between several objects (or components or, why not,systems). It has semantic to make the difference between synchronous and asynchronous communication. You may even show delay in message passing (but don't abuse of it). However for a more complex interactions between systems, you should ...


2

Publish/Subscribe would mean that any one of your micro services might want to get the GenerateReport instruction to start generating the report. If that's what you want then yes, pub/sub is part of the approach. I do however suspect that one of the micro services is responsible of coordinating the generation. In that case you are just looking for plain ...


2

The easiest solution would be to modify the subscribe function to take the id as an argument, and append the subscriber to a hashtable with the id as a key. Then the publish function would look up the id in the hashtable to get all the subscribers with that id, and publish only to those subscribers. You would have some overhead above direct calls, but ...


2

This looks like a problem for a State design pattern. Take a look at Spring statemachine. They handle listening to state change and also allows you to persist the state.


2

I believe Kafka has the notion of "Partitions". If you put messages in the same partition they will be processed in order by a single processor "By having a notion of parallelism—the partition—within the topics, Kafka is able to provide both ordering guarantees and load balancing over a pool of consumer processes. This is achieved by assigning the ...


2

The very point of asynchronous systems such as Kafka is that you gain performance in exchange for some kind of guarantees - in your case the guarantee for in-order processing. Forcing one event to wait for another pretty much removes the benefits Kafka offers to you - if you want to go that way, better use a queueing system instead of an asynchronous one ...


2

I do not have any experience with Go, but to me one of the key elements of microservices is that they communicate as loosely as possible. Eventing is a way to reduce coupling between the sender and the receiver, because (when implemented properly) the sender doesn't need to know who is listening and the receiver doesn't need to know who send it. But when ...


2

A batch interface usually means that the code is designed or optimized to process the records all at once instead of one by one at a time. Many systems have optimized methods for loading or processing batches of data. This means you get faster throughput as opposed to individual processing. Usually batch processing is done at certain intervals and is less ...


1

I'm not clear on the two options. It appears you're saying you could read directly from a queue, or let a third party write something that reads from the queue and pushes the message to you via an api endpoint. If this is the choice, I would much prefer the queue. Queues are designed to be tolerant of consumer downtime so if your consumer is down for ...


1

Fundamentally, you can have the subscribers determine when to run, letting them see every publication. Or you can have the publisher "know" somehow what subscribers are interested in which events, creating a tighter coupling than we might want. Or you can delegate this responsibility to a new third party, which might allow the decoupling you're looking ...


1

The general question is what kind of mechanism can I use to transfer data to and from publishers and subscribers where publishers or subscribers can be permanently offline? Can message queues be used for this? There is a concept of durable queues in some messaging systems, e.g., in RabbitMQ. But if your subscriber is offline for quite a long time, these ...


1

Your publisher is already tightly coupled to each subscriber, since it knows subscriber identities and expects them to be up and running. As both you and Karl Bielefeldt observe, having N distinct pubsub channels for N subscribers would make sense. In apache kafka you could use ID directly as the channel name. If you're looking for resiliency and decoupling,...


1

A subscription to a topic can be done with a POST to a topic URI. This should generate a queue that is returned in the response for the invididual subscriber. Using POST on this queue URI should create a new URI that contains a feed of items (URIs to the messages) that have to be delivered to the user. After GET on this current feed state, you can GET ...


1

First of all, you may find ready-made pub/sub solution such as AMQP or STOMP better suited to your needs than to develop you own. But if you need to rely on HTTP transport between clients and broker and don't want/need to introduce complex solution, I would prefer subscribing via long standing GET requests to defined REST endpoint. So pulling messages ...


Only top voted, non community-wiki answers of a minimum length are eligible