2

There is often introduced a concept of using metadata in events.

I just would like to understand the difference between data (which is referred to as payload) and metadata. I've read what I could find on the web on it, still would like to hear some opinions.

Let's take some general case. Say there are some items (Item is an aggregate in the system). And users (User is another aggregate too) can create and update those items.

So the question is is information about the user that created/updated the item should go to Event's data or metadata? What should be taken into consideration when deciding on it?

On one hand, user initiated the action is a candidate for metadata, but on the other hand in some view, this user may be considered a kind of important part of item's state (when it is the case), and could be treated as a part of the payload.

So just interested in a perspective on the problem in general and on the particular case.

2

The simplest way to think about metadata is that it is data that describes other data. For example, the size of the payload (in bytes) is not the content of the payload but rather a data element that is about the payload. If the payload were formatted slightly differently (e.g. extra meaningless white-space), the size might change but the content/meaning of the payload would not. The time a photograph was created and who created it is not the picture, therefore it's metadata.

The information about the user that created the data should most likely go in the metadata since it is describing the content and not actually the content that is being created.

Keep in mind that whether something is metadata or not depends on the context. Metadata is also data in itself and you can have metadata for metadata (and so on and so forth.) For example, if you were building an audit capability and you wanted to see all the users that edited the record, the list of users would be the payload in that context, not the metadata.

2

I just would like to understand the difference between data (which is referred to as payload) and metadata.

HTTP messages, with its separation of headers from the message body, might be a useful analogy to think about.

So the question is is information about the user that created/updated the item should go to Event's data or metadata?

In many cases, the answer will be "both". The representation of the information in the domain context would be part of the "body" of the message, but a copy of that same information might get lifted into the meta data to be used by generic consumers.

But as a rough guideline: if the information is part of your domain model, it will normally belong in the data, and might also be copied into the meta data. If the information is part of the plumbing, then it will normally be treated as meta data only.

I would be suspicious if our domain behaviors depended upon information available in the meta data.

1

I think this is a good question to which I don't have a straight forward answer. In short, I think the meta-data/payload distinction is incomplete.

Data, Data ... and Data

The analogy to HTTP (see VoiceOfUnreason's answer) is a good starting point and quite helpful in understanding the concept of meta data: the data that's relevant to your app is in the message body. To make the communication work, the web browser and server exchange some additional data about the message in the headers.

That's not exactly accurate, especially with today's frontends but for the purposes of data vs. meta data I think we can get away with this simplification.

At first, it seems easy to translate this into event sourcing: clearly our domain data will go into the payload, while the event number and stream id constitute meta data. After all, they essentially just exist for the infrastructure.

In practice, however, you may want to store application data, which isn't required for the ES infrastucture to work, but also doesn't affect the domain logic. Classic examples are when something happened, who initiated it, etc. So we now have three kinds of data.

Just to keep things interesting, those three classes are not necessarily always disjoint. For example, while you may want to always store who caused an event, for some events, this may actually be relevant to your domain.

Finally, your event storage & distribution infrastructure might provide special meta data fields (with additional features) for some, but not all of your application data.

Random Thoughts

As I mentioned, I do not have a fully worked out solution. I'll describe my current philoshophy below as a potential starting point.

  • I have found it useful to model an event as three separate data sections as discussed above.

    The domain model only deals with the domain part, the application layer adds application data and then publishes the events to the ES module, which adds the stream id & event numbers before storing the whole thing.

  • In CQRS, the read side may combine data from all data sections, e.g. to filter by 'who' or 'when'.

    Combining data, even from different streams, is kind of the point of the read side, so I don't see a problem here.

  • An event may contain duplicate data in some cases.

    For example, like all events, the PostAnswerEvent has an application data field caused_by with the username. Unlike many other events, it also has a domain data field author with the same user - which might be relevant in the domain for edit/voting logic.

    Now we can store exactly the domain data we actually need, but still create a generic query over everything a particular user is responsible for.

    Note however, that there could be a subtle difference between the fields. Your app logic might allow user X to create a post with author Y, if you're into that kind of thing. If not, you'd have to enforce this somehow.

  • Your event store module might further duplicate or split data to take advantage of any features your underlying ES infrastructure provides.

    That's not something the rest of your app has to be aware of.

    Essentially, if your framework has a meta-data field for user that you would like to use, move or copy the caused_by data there. Ignore whether it is called 'meta data' in your app and just use is at application data.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.