How to work with large aggregate roots?

Question

I'm learning DDD and yet I have more questions than answers.

Let's consider a model of a directory containing enormous number of files.
Here is how I see it:

Directory is an Aggregate root.
This entity should have the validation logic of checking file name uniqueness when it is added or just renamed. And File entity contains the 'SetName' logic, notifying Directory via Domain Event about name changes.
But how should Directory then work?
It is not always possible to load all files into memory. Should in this case Files repository have adhoc logic for checking name uniqueness? I suppose it is a viable decision.
However, what if some files have been already added or renamed withing current not yet commited transaction? (nothing prohibits that. Transaction boundaries are set externally in relation to business logic). Probably repository should take into account both in-memory and persisted states (merging these states can be nontrivial task.)

So, when aggregate root with all its children fits in memory - everything is fine. And as soon as you can not materialize all entities there are troubles.

I'd like to know what are the approaches for such situations. May be there is no problem at all and it is just because of my misunderstanding of the subject.

Answer 1

My answer is biased with Vaughn Vernon's Implementing Domain Driven Design great book (a must read)

1. Favor small aggregates.

If I'm to model your domain, I would model a Directory as an aggregate and File as another aggregate.

2. Reference aggregates by ids.

Therefore Directory will have a collection of FileId value objects.

3. Use factories to create aggregates.

For a simple case a factory method may be enough Directory.addFile(FileName fileName). However, for more complex cases I would use a domain factory.
The domain factory could validate that the fileName is unique using a FileRepository and a UniquefileNameValidator infrastructure service.

Why model File as a separate aggregate?

Because Directories aren't made of Files. a File is associated with a certain Directory. Also, think of a directory that has thousands of files. Loading all these objects into memory each time a directory is fetched is a performance killer.

Model your aggregates according to your use cases. If you know that there will never be more than 2-3 files in a directory then you can model them all as a single aggregate, but in my experience business rules change all the time and it pays if your model was flexible enough to accommodate the changes.

Obligatory read Effective Aggregate Design by Vaughn Vernon

Answer 2

This this is not a DDD question per se. The main question here is about synchronization context (which is here an aggregate root).

Back on topic: Directory shall block on some synchronization object of file names and perform a check whether the given file name is allowed which is O(n) in the worst case.

Answer 3

Although some may say that try to change your design, there is always a need where an AR will have a large list of simple objects. And to store them in memory is not the best thing to do from a performance perspective, however, all you need to do in such cases is to preserve the transaction boundaries. A simple solution is the following:

1. Keep the AR of the folder simple, and put a version column.
2. Have each file (let say row) reference the aggregate root, in other words, to keep the id of the folder.
3. Always perform the changes of the files, such as rename, add, remove, via the AR. In other words, if you want to add a file, load the AR, and use a method in AR to do the change. Assuming you are using a repository pattern, the addFile() will create a new file, change the version of the folder. Save those as a UoW. If someone else has changed the AR you will get an error due to Version Column (the AR version).
4. Any change to the file itself, like Editing the file or renaming it, should be done via the AR. So the version is kept in the AR. This means essentially that there are no other execution paths in your code that change the file, except the owning AR.

Some restrictions:

1. The file should belong to only one AR. If this is not the case, then model the relation of Folder -> File as containment and not the file itself.
2. You cannot move one file from one folder to the other, unless you do this change in a UoW, in other words in the same transaction. This is a special case, as you should send one request that will end up in two commands, so both ARs will be changed, therefore two new versions for each, and probably two events (file removed, file added) and this is a bit tricky because you should maintain the event orders for the two ARs which sometimes is not easy.