1

We had a discussion over this topic at the work not long ago.

We have the following situation:

  • A legacy database which is currently the base for the main operational application in our company.
  • A new content management system that should receive master data information from the mentioned database. That master data will always be owned by the legacy database, but it is needed in the CMS as a basis to construct its own data. You could think of it as categories or grouping data.
  • A trigger system in the legacy database that watches over certain tables. When an insert/update/delete occurs, it puts a notification into a database queue.
  • This database queue is watched by a (so called) loader service, which gets the data and puts it into a Kafka topic (Kafka is used for a lot of microservices in our systems).
  • A (again so called) sync service which is subscribed to the Kafka topic. Whenever something is published in the topic the service consumes it and pushes it to the CMS.

Here you can see a diagram that shows the architecture at high level:

enter image description here

For me this is (sort of) an event-driven architecture. An event happens in the legacy DB and other services get to react to it.

However, a colleague from me insisted in that it could not be considered as an event-driven architecture because the owner of the master data that provoked the event was the legacy database. Basically, my colleague's argument was that each service should own its own data and send event messages to other services (via a mediator), and that the other services should react to these events and build their data based on their own needs.

Now, I agree that each service will have to use the data from the event as it seems fit for it. But I do not agree that this argument means the architecture I explained is not event-driven. I have searched over the Internet and did not find a clear statement indicating this data ownage as being a prerequisite for an architecture to be considered as event-driven.

Is it really necessary in an event-driven architecture that each actor owns the data he works on? I.e, do I really need to not use the master data of the legacy database in the CMS? And/or should the CMS instead (ideally) create its own data and somehow map it to the legacy database data?

Some clarifications:

  • I know the presented architecture may not be the best you can have, but working with legacy systems sometimes forces you to tweak things.
  • I also know this architecture means that the services are coupled at the data level, but it is what we have for now, again due to the legacy system.
  • 4
    "Event driven architecture" is not a term protected by a trademark - at least not where I work. It can by any kind system which fits to the broad description of being "driven by events". And even if your supposed architecture would not fit to some kind of narrow definition of some buzzword, that would not make it a bad thing, so better stop overthinking this. – Doc Brown Feb 5 '18 at 19:05
  • 1
    I'm inclined to agree with Doc Brown on this one. An event-driven architecture is simply an architecture that is driven by events... nothing more, nothing less. Event-driven architectures promote decoupling: "An Event Emitter does not know the consumers of the event, it does not even know if a consumer exists, and in case it exists, it does not know how the event is used or further processed." There is no "data ownership" requirement as such. – Robert Harvey Feb 5 '18 at 19:21
  • come on you two, while most terms aren't nailed down there is some level of agreement on professional glossary. There are some things which clearly are, some which are clearly not and a large grey fuzzy area to argue about. You cant answer every question with "look within your heart for the answer" – Ewan Feb 5 '18 at 19:38
  • @DocBrown actually I agree with you, but since my colleague was so insistent on that point I just wanted to be sure I was not missing something :) – Guillem Vicens Feb 5 '18 at 20:32
  • 1
    @Ewan: Nor can one always seek enlightenment by looking in the dictionary. Unless, of course, you're looking up the definition of "enlightenment." – Robert Harvey Feb 6 '18 at 0:24
1

Event Driven Architecture

Properties:

  • Some state somewhere has changed.
  • The event represents the fact that this state has changed.
  • There is something waiting for, then receiving, and then reacting to an event.

If the system fits into that description its event-driven. It may be other things but events are driving (at least some part of) it.

Actor

You didn't go into depth on this in your question body, but i see it lurking in your questions title. I feel that this is probably where your colleague is coming from.

Properties:

  • May receive messages
  • May send messages
  • Cannot modify state it does not own.
  • Cannot read state it does not own.
  • state it owns cannot be read by a third party.
  • state it owns cannot be modified by a third party.

If it has these properties its an Actor.

I think your colleague is thinking in these terms, but has two misunderstandings:

  1. What constitutes an actor's own state.
    • A Database by definition is a separate actor. It receives some form of SQL-like message, manages its own state, sends back results, and perhaps generates other messages (events).
    • An actor may of course contain a Database actor internally, but it must do so completely, otherwise the state is shared and an actor cannot read/modify unowned state, or to state that a third-party can modify.
  2. How actors send and receive messages.
    • Your colleague appears to believe that this is only possible by some form of co-coordinator. This is not a bad solution for loosely coupled actors. But it is not the only way to organise communication.
    • Messages can come from different message-streams such as Network, GUI, Database. Each has a different message protocol, subscription, and cancellation means.
    • An actor can have more than one address on a given message stream allowing it to receive duplicates, or even receive different event sets.
    • Message passing can be accomplished through many means including lossless and lossy substrates (TCP vs UDP)
    • It is even possible to use a Reader/Writer circular buffer. While it is a shared structure, all state is clearly owned by one and only one actor at a time.
  • I think you are right regarding my colleague's approach on first point. Nice answer +1 – Guillem Vicens Jan 2 at 16:08
0

Its a tricky one, but I can see your colleagues point.

If you consider an Insert 'event' coming into the system from some user input, it does indeed seem like an event based system. Where the Input event causes action on the db and cms. But then really the owner of that data could be considered to be the user. So both systems own their own data.

But If we consider an Update, effecting a subset of columns on a single row, or perhaps many rows. The event itself depends on existing data in the DB. You couldn't really add another consumer that had not received all previous events.

In my view your setup is more like replication than an Event based system.

Additionally you further muddy the waters by using Kafka, a streaming database, rather than a message queue.

Of course it hardly matters what you call things, is there some aspect of 'event based' systems that you wish your system to have?

  • Well, regarding Kafka it is used because we have a high traffic of "events" affecting several services, plus some events need to be attended by only one suscriptors while others need to be attended by all suscriptors. Kafka gives you the power to do that (not saying there aren't others who help you there). – Guillem Vicens Feb 5 '18 at 20:29
  • What I mean is, although you can achieve (sort of) the same effect. Its a stream architecture rather than the more 'eventy' message queue. An archetypal 'event driven' system would be say, using rabbitMQ and subscribing both the DB and the CMS off the same set of messages created by whatever currently drives the database. – Ewan Feb 5 '18 at 20:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.