How to update an event log in case of new events

Question

In CQRS with Event Sourcing, if the current domain lacks a feature so it has to be extended with a new command (sent from a new saga) and an event (sent by the aggregate handler of that command), how can I deploy such feature to an already running application if the command was never sent before and so even the event was never part of the event log until the deployment.

The event is essential for the correct aggregate initialisation, so I need to have the event in the event log for every possible situation in the past where this command/event should have been sent.

Since event log should be append only. Is there a way how to add an event to the past event stream if the deployment of such feature requires it? Is it ok to do this manually? Or what is the way.

Example

To be more precise about my problem: consider a file system application, where I have aggregates for files (File) and folders (Folder). Individual files and folders (both called nodes) are aggregate roots. I cannot store the whole file system tree in a single root because of performance.

In the old version, if I create a new file in a folder, it is a command on File aggregate in which I set its parent folder (aggregate's property). Now I came to realization that I would like to have a list of IDs of nodes that a folder contains. This feature was not part of the original version so folders had no idea of the nodes that were stored in the folder.

My solution was to create a saga started by the NewFileCreated event, that would send a new AddFileToFolder command to the parent Folder aggregate. That command would apply an event (FileAddedToFolder) in Folder that would add the file's ID to its set of children nodes.

To deploy this feature, where I would like to have the correct children IDs in all folders, is restricted by the absence of these events in the original event log.

What should be the way to update the old event log in that case?

Answer 1

I think in this case instead of adding this to your aggregate, your best bet would be to have a fully-consistent local database for your aggregates to use to keep track of their relationships. It could be embedded. It could be a key/value store or relational, or whatever is convenient. This is not an eventually-consistent view model, but a separate database which is kept consistent with your aggregates.

Actually, your node relationships are in the events already, they are just not in a usable form for your delete command. An event store is not a relational database, and relationships are prohibitively expensive to query. Using a view model isn't quite good enough because it is eventually consistent. So a local fully-consistent database is your best out here. This is also a way commonly used for satisfying uniqueness checks (e.g. email address) as part of handling commands.

You can populate this database with relationships from existing events (basically a one-off denormalizer which reads events and updates this database) during deployment so it is up to date when it rolls into production.

Whenever a new file is created under a folder, this database is updated as part of handling the command.

CreateFile folderId -> FileCreated (in folderId)
                       save events
                       update local database for folderId to add child

When you go to process the delete command on the folder, you have the capability to lookup all subnodes of the folder from that database.

I question whether going event sourced is the best choice for this problem set. It's not optimal for the scenario described here. However, it still might be worth using (and overcoming issues like this) if auditability is a primary requirement.