Is Kafka an event bus or a publisher subscriber (pubsub)pattern?

Question

Building on this answer here, and its comments it entails that subscribers need to know and locate the publishers in a traditional pub/sub system. It also entails that publishers need to live at least as long as the consumers.

One of the most widely known pub/sub systems today is Kafka. However, in the case of Kafka, we have a broker, which means that it isn't a pub/sub in the above regard, but rather, an event bus (or both?) as subscribers don't need to know about publishers. Also, in the case of Kafka,the lifetime of a publisher is independent of the lifetime of a subscriber, like in event bus.

I am guessing here that since Kafka also implements ordering (i.e. commit log is sequential) it makes it a hybrid of a queue, pub/sub and an event bus patterns. Is that a correct statement?

Answer 1

I don't think the term "Event Bus" is well defined / well accepted, I have read CONTRADICTORY articles that variously state:

• Any Pub/Sub system is an event bus - if it was point to point it would be a message queue.
• An event bus has "rules" to determine how messages are processed/routed.
• A "bus" is a grouping of topic/queues.

What an Event Bus is Not.

Messaging systems (brokers) can be used for other functions such as:

• A job queue for background tasks.
• A transport for asynchronous RPC calls.

Due to the tight coupling I wouldn't typically use the term Event Bus for either of these functions.

My Definition

When I "throw around" the term Event Bus I usually mean, a pub/sub system that is publishing "events". Specifically that the sending systems is firing an "Event" (something just happened) it's not requesting any other system take action, in fact it doesn't care if anyone is subscribed.

If this was just a single topic for one type of message I would have trouble justifying that this is a "bus" so I kinda agree with the grouping argument. I don't think the rules are required, but I can certainly see use cases where messages have to be routed differently based on their content.

In short Kafka is not an Event Bus - but it can be used as a component of an Event Bus.

What is Kafka?

For that I would point you to the official website, however to avoid this response just being an RTFM, I will say that Kafka is basically a Pub/Sub system (well they use the terminology Producer and Consumer) + "Event Streaming" so you probably want to read up about Streaming to get a good idea of what Kafka is.

Answer 2

It depends what you're comparing it to.

Originally, it was used to distinguish itself from implementations where the subscribers were known to the publisher at design/compile time, i.e. the publisher effectively hardcoded all of its subscribers. This is the "oldschool" way of having one method or service call another.

In the above sense, "pub/sub" referred to a system whereby the publisher was not aware of the subscribers at design/compile time. Instead, the subscribers would add themselves to the subscription list during the publisher's runtime, as sort of a way to say "hey, let me know when there's a new thing happening please".

However, this rudimentary implementation lacked a middle man, which meant that there were certain implementation complexities such as publishers needing to be able to connect to every subscriber (causing a wide net of authorizations) and the inability to recover from an unexpected outage (e.g. a subscriber that went down or failed to handle the message from the first time).

Subsequently, event buses were created as this middle man. This changed the pub/sub game. Now, not only were publishers not aware of their subscribers at design/compile time, they weren't even aware of their subscribers at runtime either, since this is abstracted behind the event bus, as sort of a way to say "hey, I'm letting you know [this] happened, it's up to you to inform everyone who cares".

In conclusion, "pub/sub" can mean different things:

• It could refer to any implementation where the subscribers are not known to the publisher at design/compile time (i.e. when comparing to one method/service simply calling another)
• It could refer specifically to an implementation where the subscribers are not known to the publisher at design/compile time but they are known at runtime (the latter part is what distinguishes it from an event bus.

The first definition includes event buses as a form of pub/sub, the second definition intentionally excludes it.

Keep in mind that these definitions are structured by me. Maybe others prefer to think of it as everything being pub/sub and considering the existence of a broker as a secondary consideration. That's not wrong, it's just a different way of thinking about the same thing.

Is Kafka an event bus or a publisher subscriber (pubsub)pattern?

First definition says it's both; second definition says it's an event bus.