0

I have a long running process that collects together the IDs of Aggregates that have been edited/changed, so that a later command can 'publish' them to a legacy model.

I understand that I should use a kind of Saga for this which listens to certain events, then stores additional IDs for later processing.

The question I have, given I am in an event sourcing environment (using an event store for the aggregates), is how should I be expecting to store the actual saga data?

  • Should I be emitting an event from the saga to be recorded in the event store?
  • Should I have a separate persistence engine to store the data, like a MySQL instance?

I am confused as to how complicated it would be for option 1, but also, to have to create another data schema with a single table for option two seems overkill also.

What are others in the community doing in this kind of scenario?

  • how/when do you create your Process; when does the Process run? – Constantin Galbenu Jun 5 '18 at 12:59
  • Maybe I used the wrong terminology. It's more like an aggregate that collects together the IDs of another type of aggregate on any event that is emitted from it. When another 'final' event is triggered, it process the list of aggregates from their IDs into a legacy read model. There is no real single start point. – designermonkey Jun 5 '18 at 13:16
  • So, you have a process (let's use this term) that collects Aggregate IDs and when something happen (like a PublishedEvent) you need to perform some action that has side effects, then the process ends. Am I right? – Constantin Galbenu Jun 5 '18 at 13:18
  • Yep, more or less :) – designermonkey Jun 5 '18 at 13:18
  • It has to be persisted somewhere though, it can't be stored in memory as we use PHP with it's single request -> response cycle. – designermonkey Jun 5 '18 at 13:19
1

You have two options to run a Saga:

  1. Synchronously/in-process, at each request, after the Event store persists the events, you sent them to the subscribed Sagas. This has the disadvantage that when something wrong will happen (i.e. server is restarted) you will lose events.

  2. Asynchronously, as a long running standalone process that tails/polls the Event store, process the new events, acknowledges them in a persistent storage and then tails/polls again. Unlike the first option, when something wrong happens you just restart the process from the last acknowledged event. This is the preferred way, when possible.

From what I understand from your comments, you "have no mechanism to have a continually running process" so you are left with option no.1.

A Saga models a long running process; in Event sourcing this means a process that spans multiple events. In your case the Saga must collect the Aggregate IDs and persist them as if it were an Entity. For this you can use a flat persistence (as opposed to an event-sourced one), a simple NoSQL database. You should take care about concurrent modification of the Saga entity as events could come at the same time; for this you could use low level database operations like MongoDB's addToSet.

From the DDD point of view, this Saga entity is in fact a new Aggregate type, i.e. PublishSomeAggregatesToLegacySystemAggregate, that lives maybe in another Bounded context. The Saga would just react to events and send commands to ProcessSomeIDsAggregate, just like any other Saga. This new Aggregate should be non-event-sourced. So, when an Aggregate is edited/changed, the Saga sends a AddAggregateToBePublishedToLegacySystem command to the PublishSomeAggregatesToLegacySystemAggregate. When the final event is received by the Saga, it will send the PublishTheAggregates command.

You should be careful to what kind of logic you put inside the Saga. It should contain only coordination logic, not low level like SQL or legacy system access code.

0

"Saga", in this context, is more commonly referred to as a "process manager", in part because the original meaning of saga was different, and in part because we suck at naming things.

My recommended starting point on process managers was written by Rinat Abdullin.

Process managers are, essentially, command recommenders; given that I know about events A, B, and C, I recommend commands Y and Z. The are a book keeping device to keep track of work that should be done in the system.

They often have an internal expression as a state machine.

state = fsm.start()
           .andThen(A)
           .andThen(B)
           .andThen(C)
           .state()

pendingCommands = state.commands()

As a book keeping device, they are a lot like "views", in the sense. Like views, if you have a history of all of the events available already, you can rediscover what state a process is in.

For instance, you might imagine a stand along service, that has the code that defines all of your processes. On start, it pulls from your event store the entire history of the world, computes in memory the state of each managed process, and sends the command messages to your domain model. On each clock tick after that, it checks for new messages, reads them in, and sends new command messages.

If the service panics, no problem -- just launch a new copy that starts all over again.

So in a sense, you don't need to write anything down at all.

(Which isn't to say that you shouldn't -- there are advantages to knowing when things happened, for instance, or being able to reproduce the behavior of the production system in a lab environment.)

  • Thanks for explaining a little more, unfortunately, I have a PHP system that has to store the state of each saga between request response cycles, and that this process can be collecting the IDs for weeks. We have no mechanism to have a continually running process, so my question is still the same, how should I store the data? – designermonkey Jun 5 '18 at 12:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.