1

I am intrigued by the Event Sourcing pattern but am struggling to design an event sourcing model and put it in concrete form.

First off, I want to make sure that I understand the main use cases of this pattern. As I understand it, one uses event sourcing is to capture state changes in your app at different points in time so that you can:

  • Replay events to rewind your app to a previous state; or
  • Replay events in another environment to recreate production at some point in time and troubleshoot it

So first off, if I have misunderstood the main use case(s) for event sourcing, or if I am missing any major ones, please begin by correcting me!


Assuming I'm more or less on track, my real mental roadblock lies with the event persistence model.

  • How many databases are there in an event sourcing system? Are there two databases (one for storing main entities, and another for storing the events)? Or is there just one database where all entities are the persisted events themselves?
  • How do you capture events (and persist them!) for things that are totally outside the app's control? For example: the state of some 3rd party API that your app integrates with, or a message broker (and the state of all its messages/queues) that your app integrates with, runtime command-line arguments passed into your app, etc.?
  • How does the replay mechanism actually work? So you have some entities. You have events describing state changes to those entities. You can read those events (and/or the entities they represent) out of the database, but then you need something to actually write those events somewhere else. So does this "replayer" just treat each subsequent event as an update to an existing entity?

My best attempt (thus far):

The entity(ies):

// Groovy pseudo-code
class Order {
    Long id
    Long userId
    Long paymentMethodId    // Perhaps an ID to a 3rd party system that
                            // is PCI compliant, etc.

    // Constructors, getters/setters, etc.
}

// Each 'Order' has 1+ 'OrderLineItems'
class OrderLineItem {
    Long id
    Long orderId
    Long productId    // 'Product' is yet another entity, etc.
    Integer quantity

    // Constructors, getters/setters, etc.
}

The event:

abstract class BaseEvent<ENTITY> {
    Date occurredOn
    Date recordedOn
    ENTITY entity

    BaseEvent(ENTITY entity) {
        super()

        this.occurredOn = new Date() // now
        this.entity = entity
    }
}

class OrderEvent extends BaseEvent<Order> {
    // Perhaps some other metadata about order events

    OrderEvent(Order order) {
        super(order)
    }
}

OK so this is a good step in the right direction, but still doesn't help me figure out how the "replay mechanism" will be able to load OrderEvents out of a database, and actually "replay them" to put another database into a particular state representing a single point in time.

  • Event sourcing is a persistence model. You store your entities in the form of a stream of events. The stream of events is the source of truth; you rehydrate an entity by replaying its event stream from the beginning. – Benjamin Hodgson Nov 7 '15 at 9:33
  • Thank you for the copy pasta from Google, @BenjaminHodgson. Still neither helps me nor answers my question. – smeeb Nov 7 '15 at 10:42
  • Um, I wrote those words myself. No need to be rude. – Benjamin Hodgson Nov 7 '15 at 11:17
2
+100

I want to make sure that I understand the main use cases of this pattern.

Another nice use case described by Fowler is distributed storage. A cluster of systems with in-memory databases are kept up to date with each other through a stream of events.

How many databases are there in an event sourcing system? Are there two databases (one for storing main entities, and another for storing the events)? Or is there just one database where all entities are the persisted events themselves?

Main entities and events are separate objects and both are persisted, either in the same or in different databases.

How do you capture events (and persist them!) for things that are totally outside the app's control?

You'll need to wrap any external systems with a gateway. The gateway must be able deal with any replay processing that the Event Sourcing system is doing. See sections 'External Updates/Queries/Interactions' of Fowler's description of this pattern.

How does the replay mechanism actually work?

Take Fowler's most basic example:

enter image description here

Each event object has a 'process' method to perform the required update. A replay works like this:

for each Shipping Event e, in ascending order of e.occurred:
    e.process();
  • Thanks @www.admiraalit.nl (+1) - I was getting ready to give up on Event Sourcing! What you're saying absolutely makes sense, I just have a quick followup question for you if you don't mind, mainly just to confirm that I "get" Event Sourcing. Let's use the ArrivalEvent object from your image above. It's process() method sets ship.location = port...I completely understand this part. – smeeb Nov 11 '15 at 10:35
  • But my understanding of Event Sourcing is that its general purpose is to reconstruct state at a given point in time. So in a real world use case, I would imagine we would then want to persist ship (with the updated port location) to an actual (non-production) DB. So can I assume that a "real world" ArrivalEvent would also contain DAOs for persisting the newly-updated ship instance? – smeeb Nov 11 '15 at 10:35
  • 1
    Possibly, but I would rather expect that the ship itself would be the DAO (or inherit from, or contain the DAO). I think the pattern itself does not prescribe which object is responsible for persistance. – www.admiraalit.nl Nov 11 '15 at 11:57
1

How many databases are there in an event sourcing system?

Does not really matter. You can have everything in a single database or split them out, just like you could split the data for an online store into a database with users and a database with products if you wanted to.

How do you capture events (and persist them!) for things that are totally outside the app's control?

Persisting the state of queues is not something you would do, unless I'm misunderstanding your question. As for the 3rd party API or command-line, you simply capture an event at the point that it occurs.

If you have a 3rd party API that you need to check (for example, query the exact price of a product at the time the order is placed) you either work with it in replay if the API can handle it, or store the data from the external party so you have all the needed data in your event. For this example, you would either query the API for the price at the time the event was recorded, or store the price in your event so you don't need the external API.

How does the replay mechanism actually work?

You have events operate on domain objects. If you have a list of events related to an Order you would allow each event to operate on the Order they are related to. Remember that a stored event is basically just a bunch of data. Using that data, your objects can then redo the actions they did.

  • Thanks @JDT (+1) - a few followups if you don't mind: (1) I think this would make a lot more sense if I could see actual code. All the online searches I've done for "event sourcing examples" just turn up blogs and diagrams, but no concrete code. Do you know where I might see event sourcing code that shows: (a) the entity (say, Order), (b) the event (say, OrderEvent) and (c) the replay mechanism iterating through OrderEvents and updating the state of a given Order? If you don't know of any online/readily available code examples, would you mind updating your answer with some? – smeeb Nov 5 '15 at 12:20
  • And (2) just curious, any clue as to why it was coined as event "sourcing"? Guess I'm hung up on the "sourcing" part. To me, it's more like event snapshotting. Thanks again! – smeeb Nov 5 '15 at 12:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.