What is the relationship between Event Store and Persistence Mechanism when designing Aggregate + Event Sourcing?

Question

I'm trying to implement an Event Store, to be used with Aggregate+Event Sourcing (A+ES), and I've run into I think a gap in understanding.

I'm reading Implementing Domain Driven Design, using both the Domain Event/Event Store and Appendix A: A+ES sections as a template/guide.

In the book, there are two types used, the Event Store, and an IAppendOnlyStore. The examples are in C#.

The book goes on to say

As illustrated in Figure A.14, the class implementing IEventStore is a project-specific wrapper around the more generic and reusable IAppendOnlyStore. While the IEventStore implementation deals with serialization and strong typing, the IAppendOnlyStore implementations provide low-level access to various storage engines.

Implementing Domain Driven Design (Vaughn, p. 563)

Different IAppendOnlyStore implementations can exist. Some examples the book gives are File-based, MS SQL Server, MySQL, Windows Azure Blob, and MongoDB.

The IAppendOnlyStore contains methods that operate on byte[] objects. According to the book, the Event Store is responsible for serializing and deserializing a Domain Event into a byte[]. The Event Store then passes the serialized byte[] to the IAppendOnlyStore, which then persists the byte[] into the persistence store. The IAppendOnlyStore also has methods for retrieving records from the persistence store. Whether appending or retrieving, a unique identifier is used to indicate which records to retrieve, or which event stream to persist to.

Here's where the gap in my understanding comes in. The example implementation of Relational Persistence in the book uses MySQL. The MySQL IAppendOnlyStore implementation persists the events in a table, that has columns (Id, Name, Version, Data). Id is a unique, persistence local identity for the record, Name is the Aggregate Identity, Version is the version of the event stream, and Data is a blob, storing the serialized Domain Event. The implementation opens a connection to the database, executes a command, passing in the parameters, and commits the transaction. The IAppendOnlyStore implementation knows intimate details of the persistence mechanism. For a File-based implementation, it would probably know where the file is; for Azure, it would probably know credentials; for MongoDB it would probably know about the documents required.

The gap is that this seems like a leaky abstraction detail.

It feels like all of those intimate implementation details should be completely encapsulated within the IAppendOnlyStore implementation. It feels like only the implementation should know that the data is stored in a blob column in MySQL, and would therefore need to be serialized. However, this leaks out of the implementation into the Event Store. The responsibility of serializing and deserializing the Domain Event exists in the Event Store, not the IAppendOnlyStore.

This feels like the Event Store either knows, or is dictating, that the underlying persistence mechanism will store the Domain Event as a byte[]. This feels wrong to me. If we wanted to persist the Domain Event properties as separate columns, in a normalized form in the database, the IAppendOnlyStore implementation should support that. It already supports different types of connections; it knows whether it's executing a raw query or a stored procedure; whether one table is updated or multiple; so it should have the freedom to interact with the underlying mechanism in whatever way is necessary. This, to me, implies that it is actually the responsibility of the IAppendOnlyStore to serialize the Domain Event, not the Event Store, because the IAppendOnlyStore is the only component that knows, or should know, what the underlying persistence mechanism requires, in terms of format, datatype, and data shape. In this case, that the Domain Event is serialized as a byte[], and stored in a single blob column, in a single table. I'll call this Assumption One.

With Assumption One in mind, it is also implied that deserialization is the responsibility of the IAppendOnlyStore, as it is the opposite pairing of serialization. If the IAppendOnlyStore is responsible for serializing an in-memory Domain Event instance, into a byte[] for the underlying persistence mechanism, it seems reasonable that it would also have the responsibility of deserializing a byte[] from the underlying persistence mechanism, into an in-memory Domain Event instance. I'll call this Assumption Two.

With Assumption Two in mind, all transformation from in-memory object, to persisted object, and back, is the responsibility of the IAppendOnlyStore. Additionally, IAppendOnlyStore supports retrieving all, or some, events associated with a given Aggregate Identity. In the example in the book, the Event Store translates an IEnumerable<DataWithVersion> from the IAppendOnlyStore, into an EventStream containing a list of the deserialized events. With Assumption One and Two in mind, this would not be an operation on the Event Store, as that responsibility would lie with the IAppendOnlyStore.

So from one single feeling, that the IAppendOnlyStore should have exclusive knowledge of how a Domain Event is persisted, nearly all responsibility has been removed from the Event Store. It doesn't deal with serialization; deserialization; enforcing strong typing; retrieving events, or appending them; translating from persistence object to in-memory object, or vice-versa. At this level, it's delegating all of those concerns to the IAppendOnlyStore. All that's left is concurrency protection, but even that seems more like a concern for the storage mechanism, than the Event Store.

Did I miss something? At this point, the only reason I can think of that would justify the existence of the Event Store, is to provide a non-storage/non-persistence/non-lowest-level oriented interface that indicates that an event is stored somewhere, somehow. It feels like its only purpose is to delegate calls to whatever instance of IAppendOnlyStore it's been told to use, to relieve the client of knowing that it is interacting directly with the persistence mechanism.

Is there some other reason to keep these responsibilities in the Event Store that I missed from my reading of the book? Should the Domain Events really be serialized in the Event Store, as opposed to the persistence mechanism? Is it preferable for the Event Store to guarantee or enforce that the underlying mechanism persists events as serialized binary blobs?

Answer 1

Did I miss something?

I think you've missed a couple things.

Highest on my list: your event storage client is plumbing. It's a mistake to get overly invested in this design vs that design - the business isn't going to care about those details so long as everything works.

The piece that's critically important is making sure that today's code can read yesterday's events. That means being able to test the serialization logic, and you'll probably want to be able to run those tests without dependencies on the network.

But the rest of it is I/O, and unless your project is catastrophically successful, tinkering with your I/O designs isn't going to pay dividends. It's much more likely that you are going to write the code, bang on it with a hammer until it is good enough, and then leave it alone until you are forced to migrate your data to a different storage appliance.

Note: that also means that if you want to implement this as one abstraction instead of two, that's probably going to be fine.

Parnas 1971 would encourage you to use multiple abstractions because information hiding. Which is to say, if you need to protect your serialization code from changes to where you put the bytes, or vice versa, then you will want at least one information hiding boundary between the two.

It might also be useful to review Mark Seemann's work on Dependency Rejection; I/O code lives in the imperative shell anyway, it might make sense to treat your event repository as one enormous I/O component into which you can inject the (well tested) functions that are responsible for serialization and deserialization of events.

Design is what we do to get more of what we want than we would get by just doing it -- Ruth Malan.

Ultimately, to get a clean design you are going to need a clear picture of what your "what we want" is; if it is different from someone else's, then your designs probably won't be the same.