How to implement a process manager in event sourcing

Question

I'm working on a small example application to learn the concepts of CQRS and event sourcing. I have a Basket aggregate and a Product aggregate which should work independently.

Here's some pseudo code to show the implementation

Basket { BasketId; OrderLines; Address; }

// basket events
BasketCreated { BasketId; }
ItemAdded { BasketId; ProductId; Quantity }
AddItemSucceeded { BasketId; ProductId; Quantity }
AddItemRevoked { BasketId; ProductId; Quantity }
ItemRemoved { BasketId; ProductId; Quantity }
CheckedOut { BasketId; Address }

Product { ProductId; Name; Price; }

// product events
ProductReserved { ProductId; Quantity }
ProductReservationFailed { ProductId; Quantity }
ProductReservationCancelled { ProductId; Quantity; }

Commands are pretty similar to the events, using the imperative name and not past tense.

Right now these work just fine independently. I issue a command AddItem, and it creates a ItemAdded event on the Basket aggregate which does what it needs to do with the state of the 'Basket'. Similarly, for product the command and events work just fine.

I'd now like to combine this into a process which would go something like this (in terms of commands and events that happen):

The process manager would do the following:

on BasketCreated: CreateShoppingProcess
on ItemAdded: ReserveProduct
on ProductReserved: SucceedAddingItem // does nothing, but needs to be there so that the basket knows it can check out
on ProductReservationFailed: RevokeAddItem
on RemoveItem: CancelProductReservation
on Checkout: CreateOrder // create an order and so on...

The questions that I couldn't find definitive answers to are:

1. Do I need to persist the process manager? It seems like I do, but I'm not sure
2. If I do, I need to save the events for the process manager. However, the events that It's listening to are tied to the aggregates. Do I add the process id to those? Do I have separate events just for the process manager? How to do this and keep as DRY as possible
3. How do I know what basket the ProductReserved events are for? Is it OK to have a BasketId on those too, or is that leaking info?
4. How do I keep a relationship between events, how do I know which ItemAdded produced which ProductReserved event? Do I pass along an EventId? This seems odd...
5. Should I implement the Basket as a process manager instead of a simple aggregate?

After some more research I came to this: A Saga is something that keeps its own events and listens to events from the outside. Basically, it's an Aggregate that can also react to events happening outside it's own little world.

A Process Manager works with the events from the outside and sends out commands. It's history can be rebuilt from the events that have happened on the Aggregates which share a common identifier like a correlationId.

Answer 1

Review what Rinat Abdullin wrote about evolving business process. In particular, notice his recommendation for developing a business process in a fast changing environment -- a process manager is "just" an automated replacement for a human being staring at a screen.

My own mental model of a process manager is that it is an event sourced projection that you can query for a list of pending commands.

Do I need to persist the process manager? It seems like I do, but I'm not sure

It's a read model. You can rebuild the process manager from the history of events each time you need it; or you can treat it like a snapshot that you update.

If I do, I need to save the events for the process manager.

No - the process manager is a manager. It doesn't do anything useful on its own; instead it tells aggregates to do work (ie, make changes to the book of record).

How do I know what basket the ProductReserved events are for? Is it OK to have a BasketId on those too, or is that leaking info

Sure. Note: in most "real" shopping domains, you wouldn't insist on reserving inventory before processing an order; it adds unnecessary contention to the business. It's more likely that your business would want to accept the order, then apologize in the rare case that the order can't be fulfilled in the required time.

How do I keep a relationship between events, how do I know which ItemAdded produced which ProductReserved event?

Messages have meta data - in particular, you can include a causationIdentifier (so you can identify which commands produced which events) and a correlationIdentifier, to generally track the conversation.

For instance, the process manager writes its own id as the correlationId in the command; the events produced by a copy the correlation id of the command, and your process manager subscribes to all events that have its own correlationId.

Should I implement the Basket as a process manager instead of a simple aggregate?

My recommendation is no. But Udi Dahan has a different opinion that you should review; which is that CQRS only makes sense if your aggregates are sagas -- Udi used saga in the place where process manager has become the dominant spelling.

should process managers retrieve aggregates?

Not really? Process managers are primarily concerned with orchestration, not domain state. An instance of a process will have "state", in the form of a history of all of the events that they have observed -- the correct thing to do in response to event Z depends on whether or not we have seen events X and Y. So you may need to be able to store and load a representation of that state (which could be something flat, or could be the history of observed events).

(I say "not really" because aggregate is defined vaguely enough that it's not completely wrong to claim that list of observed events is an "aggregate". The differences are more semantic than implementation -- we load process state and then decide what messages to send to the parts of the system responsible for domain state. There's a bit of hand waving going on here.)

So the PM does not need to use one type of state management over another because it is only responsible for doing stuff live and never during replays?

Not quite - state management isn't a do-er, it's a keeper tracker of-er. In circumstances where the process manger shouldn't emit any signals, you give it inert connections to the world. In other words, dispatch(command) is a no-op.

Answer 2

What you're looking for has a pattern called a "Saga", which is essentially a process manager.

Saga's are perfect for long running processes too, because it can maintain state between correlated commands.

Here is a great article on Sagas