Persistence models for Event Sourcing

Question

I am intrigued by the Event Sourcing pattern but am struggling to design an event sourcing model and put it in concrete form.

First off, I want to make sure that I understand the main use cases of this pattern. As I understand it, one uses event sourcing is to capture state changes in your app at different points in time so that you can:

• Replay events to rewind your app to a previous state; or
• Replay events in another environment to recreate production at some point in time and troubleshoot it

So first off, if I have misunderstood the main use case(s) for event sourcing, or if I am missing any major ones, please begin by correcting me!

---

Assuming I'm more or less on track, my real mental roadblock lies with the event persistence model.

• How many databases are there in an event sourcing system? Are there two databases (one for storing main entities, and another for storing the events)? Or is there just one database where all entities are the persisted events themselves?
• How do you capture events (and persist them!) for things that are totally outside the app's control? For example: the state of some 3rd party API that your app integrates with, or a message broker (and the state of all its messages/queues) that your app integrates with, runtime command-line arguments passed into your app, etc.?
• How does the replay mechanism actually work? So you have some entities. You have events describing state changes to those entities. You can read those events (and/or the entities they represent) out of the database, but then you need something to actually write those events somewhere else. So does this "replayer" just treat each subsequent event as an update to an existing entity?

---

My best attempt (thus far):

The entity(ies):

// Groovy pseudo-code
class Order {
    Long id
    Long userId
    Long paymentMethodId    // Perhaps an ID to a 3rd party system that
                            // is PCI compliant, etc.

    // Constructors, getters/setters, etc.
}

// Each 'Order' has 1+ 'OrderLineItems'
class OrderLineItem {
    Long id
    Long orderId
    Long productId    // 'Product' is yet another entity, etc.
    Integer quantity

    // Constructors, getters/setters, etc.
}

The event:

abstract class BaseEvent<ENTITY> {
    Date occurredOn
    Date recordedOn
    ENTITY entity

    BaseEvent(ENTITY entity) {
        super()

        this.occurredOn = new Date() // now
        this.entity = entity
    }
}

class OrderEvent extends BaseEvent<Order> {
    // Perhaps some other metadata about order events

    OrderEvent(Order order) {
        super(order)
    }
}

OK so this is a good step in the right direction, but still doesn't help me figure out how the "replay mechanism" will be able to load OrderEvents out of a database, and actually "replay them" to put another database into a particular state representing a single point in time.

Answer 1

I want to make sure that I understand the main use cases of this pattern.

Another nice use case described by Fowler is distributed storage. A cluster of systems with in-memory databases are kept up to date with each other through a stream of events.

How many databases are there in an event sourcing system? Are there two databases (one for storing main entities, and another for storing the events)? Or is there just one database where all entities are the persisted events themselves?

Main entities and events are separate objects and both are persisted, either in the same or in different databases.

How do you capture events (and persist them!) for things that are totally outside the app's control?

You'll need to wrap any external systems with a gateway. The gateway must be able deal with any replay processing that the Event Sourcing system is doing. See sections 'External Updates/Queries/Interactions' of Fowler's description of this pattern.

How does the replay mechanism actually work?

Take Fowler's most basic example:

Each event object has a 'process' method to perform the required update. A replay works like this:

for each Shipping Event e, in ascending order of e.occurred:
    e.process();

Answer 2

How many databases are there in an event sourcing system?

Does not really matter. You can have everything in a single database or split them out, just like you could split the data for an online store into a database with users and a database with products if you wanted to.

How do you capture events (and persist them!) for things that are totally outside the app's control?

Persisting the state of queues is not something you would do, unless I'm misunderstanding your question. As for the 3rd party API or command-line, you simply capture an event at the point that it occurs.

If you have a 3rd party API that you need to check (for example, query the exact price of a product at the time the order is placed) you either work with it in replay if the API can handle it, or store the data from the external party so you have all the needed data in your event. For this example, you would either query the API for the price at the time the event was recorded, or store the price in your event so you don't need the external API.

How does the replay mechanism actually work?

You have events operate on domain objects. If you have a list of events related to an Order you would allow each event to operate on the Order they are related to. Remember that a stored event is basically just a bunch of data. Using that data, your objects can then redo the actions they did.