Merging aggregates with Event sourcing

Question

I'm currently evaluating Event Sourcing and CQRS for an implementation of a new business requirement at my day job. While I can't really speak about the actual business problem, I can give a few reasons for why we think that Event Sourcing might be a good fit:

• great auditing capabilities based on the history of events
• "travelling back in time" to recreate a previous state of an aggregate (e.g. for debugging purposes)
• the ability to create new projections that take the full history into account

Since I can't go into detail about the exact domain we're in, I will describe my problem using the domain described in this Kata dealing with quiz games.

I think I got the general idea of Event Sourcing and how CQRS links to it. However, all examples I can find use domains with clear separations between aggregates as well as between different instances of the same aggregate (in the Kata mentioned above, quizzes and games have a clear relationship. There's no interdependence between different quizzes or different games).

The problem

In my case I have the problem that it must be possible to merge different instances of the same aggregate (in our sample domain this could mean that it must be possible to merge different quizzes together into one quiz) as well as undoing this merge later on (reconstructing the two original quizzes from the merged one).

This constraint adds quite some complexity when it comes to constructing the current state of an aggregate, because it's necessary to read the whole event stream from the beginning to be sure that all relevant events are taken into consideration. It's not possible to partition the event stream in a useful way because it's impossible to tell which aggregates will be merged later in the future. It might even be a problem when the event stream gets partitioned, because the temporal order of related events gets lost.
From what I understand, partitioning the event stream allows for a fast provision of the events that are necessary to build up the current state of an aggregate. For instance, if I want to know the current state of the quiz with ID 124ecf, I technically could filter the event streams to just have the events for this exact ID which would drastically reduce the number of events. If this is not possible, like in my case, reading the event stream ad hoc to recreate the state of an aggregate will become very slow and impractical over time.

The solution I came up with so far

The only solution for this problem that seems to be possible to me is to work with rolling snapshots for all necessary projections. The snapshots would update themselves continuously, building up a state optimized for their specific use case (processing commands, answering queries etc.).
I'm skeptical about this idea, because it requires quite some effort. Most of the implementations of typical applications don't require rolling snapshots for most use cases because building up the desired state from the event stream is fast enough. This simplicity is lost in my case.

The question

My question could be split up in several parts:

• Is it a good idea to use Event Sourcing for domains like these where it's not possible to draw clear boundaries between different instances of the same aggregate?
• Does it make sense to heavily rely on using rolling snapshots to get the desired performance?
• Is there another way other than rolling snapshots to implement this?
• I can't think of a way for partitioning the event stream. Am I missing something? Are there some techniques that allow partitioning/sharding under the given circumstances?

Answer 1

I don't really know the details of why you are doing Event Sourcing, so I can't comment on whether it is a good fit.

However, if I understood you correctly, you basically have a hierarchical event stream that describes your objects. So instead of storing events in a linear fashion, like in most (all?) examples, maybe you could store them hierarchically, like in a graph database.

So reconstructing a state becomes a depth-first walk of all parents (or children, depending on how you look at it). You can also get all direct parents of a state to "undo" the state.

To be honest, I don't know anybody who has done something like that, but it is worth a proof-of-concept I think.

Answer 2

Event sourcing is not business logic

Your problem description is a bit all over the place. At times you discuss a merge of quizzes, other times you seem to be discussing a combination. I suspect you've either not analyzed this enough, or have overanalyzed it to the point of losing touch with the main objective.

In all cases I can reasonably think of, this is not a problem that you solve by changing your event sourcing. It's a domain (= business logic) problem. I suspect you are muddying the separation that you should maintain between DAL logic and BLL logic. Your domain shouldn't know (nor care) whether your data store uses event sourcing or not.

---

Copy or reference?

Suppose you merge quiz A into quiz B. If I now add a question to quiz A, are you expecting this new question to automagically appear in B?

If yes, then you really need to analyze that part of the behavior because you've got more and bigger fish to fry to implement this. In this case I suggest you

If not, then I don't see what the complexity of the question is or why merging would be an issue. Merging A into B would then effectively the same as adding multiple questions to B (or a series of actions, each of which adds a question to B).

Overzealous blocking?

I have to make sure that I process all events in the correct temporal order to make sure that there were no "merge B into A" events in the stream for B before "merge A into B" happened in the stream for A.

That would be an business logic decision, there's no inherent problem with allowing quizzes to merge into each other.

I'm also not sure your intended logic actually applies to the cases you're trying to handle. Using the following scenario:

• A contains questions 1,2,3
• B contains question 4
• I merge A into B, B now contains 1,2,3,4
• Over time, the content of A gets changed and A now contains questions 5,11,13 (nothing else)

Why am I now not allowed to merge B into A? I see no reason to exclude this behavior.

I suspect you might be trying to prevent merges from creating duplicates, but that can be resolved using very different (and simpler) methods:

• Simply allow duplicates and leave it up to the user to clean it
• When merging, filter out the questions that would be duplicates and only add the non-duplicates
• Refuse to merge if the source and destination have any shared question, return a meaningful error to the user

Note: how you define equality between two questions (which is how you know something is a duplicate or not) is up to you. It could be an ID check, it could be a direct match on the question text, ...

These solutions cover essentially every angle, without needing you to break your head over the event streams.

Merging or combining?

merge different instances of the same aggregate (in our sample domain this could mean that it must be possible to merge different quizzes together into one quiz) as well as undoing this merge later on (reconstructing the two original quizzes from the merged one)

This doesn't sound like "merging A into B", it sounds like "making C which is equal to A+B".

The same question as before repeats itself, if you adjust A or B after having made C, do you expect C to update itself?

If yes, then you should create a new "combined quiz" type which doesn't contain questions (like normal quizzes do), just references to other (sub)quizzes. C would then hold a reference to both A and B, so it acts as if it's a grouping of the two.
Note: If/how you handle duplicate question is up to you.

If no, then C is really just a regular quiz, and when you create it, you automatically add the questions from A and B to this new C quiz. After creation, the A, B and C quizzes each live their own life and are no longer connected.

In either case, this would make it very easy to reconstruct the original quizzes, since you never had to alter A or B at any point during this process.