Is the use of a message broker aligned with Sam Newman's Building Microservices?

Question

In one of the projects the CTO has chosen to use a message broker in order to connect microservices. Is the use of such software aligned with the microservices theory?

Attempt to answer the question

In Sam Newman's Building Microservices chapter 12: Bringing it All Together, the author mentions seven principles including Decentralize all the things. The last paragraph of that part is as follows:

This principle can apply to architecture too. Avoid approaches like enterprise service bus or orchestration systems, which can lead to centralization of business logic and dumb services. Instead, prefer choreography over orchestration and dumb middleware, with smart endpoints to ensure you that you keep associated logic and data within service boundaries, helping keep things cohesive.

Enterprise service bus

implements a communication system between mutually interacting software applications in a service-oriented architecture (SOA)

Orchestration systems

the automated arrangement, coordination, and management of computer systems, middleware, and services

Choreography

the art or practice of designing sequences of movements of physical bodies (or their depictions) in which motion, form, or both are specified

Dumb middleware

I’m pretty confident that there exist enterprise service busses which are handling similar scenarios and those are the opposite of simple “dumb” ZeroMQ-like message passing frameworks.

Smart endpoints

Q: And regarding smart endpoints.. if I get it right, Microservice=endpoint, sort of something that can send/receive messages. The reason the endpoint is smart, because it has a logic inside, not on the middleware(e.g. ESB). Right? A: Exactly, endpoints have the logic and I actually did a project in an open source team which used JMS as the underlying communication for an ESB, so it should be still rather dumb

Questions

1. What is the definition of choreography in IT?
2. What are examples of dumb middleware?
3. Are microservices smart endpoints?
4. Is the use of message brokers, e.g. kafka, rabbitmq, zeromq and qpid in Microservices a bad practice?

Answer 1

Okay, I'll try here.

Firstly Sam's opinion is considered the normal way to "do" micro-services, but there are trade-offs.

So:

1. Choreography means the actors act independently but according to a shared set of instructions. This is as opposed to orchestration where there is a central conductor that tells the players what to do and when to do it.

   Orchestration is commonly associated with ESB's which hold a lot of the business logic (think scripts and choices), and distribute work to the services.

   Choreography is more commonly associated with event-driven systems, where services broadcast events to the world, and interested parties act on those events independently.

2. Dumb middleware implies that the middleware does not actively react to the content of the messages it transports. It routes "stuff" from A to B based on the address on the envelope. Most messaging middleware is dumb in this respect, though there is often a degree of intelligence added to provide delivery guarantees e.g. exactly-once. This is, arguably, not intelligence per se, but still is, in my opinion, a bit too smart due to transactional concerns, and failure handling semantics, but lets not go there just yet. Trust me when I say that architects argue about this stuff all the time, and Sam falls on the side of light and truth here, if not convenience and simplicity.

3. If you have dumb middleware, then micro-services absolutely have to be smart end-points, otherwise you've got a brittle (tends to break easily) system of the worst kind. How much smarts depends on how you want to deal with overload and failure. Have a look at all the goodies used by Netflix to add these smarts to micro-services in a reusable way e.g. Hystrix.

4. No. As much as middleware should not be too clever, it shouldn't be stupid. Kafka, ZeroMQ and so on are highly sophisticated sub-systems when you want to do event-driven systems correctly. That doesn't mean you have to use them, but they have some useful features. Most large-scale systems do both. Synchronous stuff is done using http request/response. Asynchronous stuff uses messaging. Messaging can be centralised (Kafka) or distributed (local ZeroMQ), but the centralised ones can easily insulate you from service failure/recovery.