Should I create one large initial event or break down in smaller events?

Question

I'm experimenting with event sourcing for an application we haven't build yet. No, I won't implement this without any thought, I'm just experimenting.

My domain model looks somewhat like this. The whole model is created at once via a REST service call by external parties. Please note that the status history entities are not event-sourcing related. These come from some workflow engine we are using.

After that point, there are smaller changes to the model. some details are changed. Some status history is added. Sometimes Invoice items are added. Several applications change different aspects to this model, but not as invasive as the original creation of the model. Also there are several application that need to know details about the model, but in different forms. This is what led me to believe event-sourcing could be a good fit.

My main question: Should the first creation of my model be one big event that says "Invoice bundle created" including all nested details needed for creation (in JSON or other form). Or should i break up this initial event in several smaller events, somewhat like the picture below.

My second question: how should I deal with the files delivered? Should I store those as blob storage and refer to them from events or should I include those as base64 string in my events?

Answer 1

I think the answer to your question comes from defining what the failure states for your application are. Event based systems come in a couple flavors: persistent queues and objects passed in memory. Both of those have different types of failure states. For example, persistent queues write the event to disk until a client confirms receipt. Objects passed in memory no longer exist if the process dies.

Impacts of large messages:

• Potentially can fill up a small drive with persistent queues
• Increases the memory footprint in your process
• Impact of a lost message includes all the content of the message

It's more efficient to store data properly and simply reference it in your messages. That helps minimize impact to your persistent queue and the memory footprint your application has. Both of those mean you can run your application with less memory, and there is less chance for your event delivery system to get clogged.

As to whether you use one message or several depends on what your consistency needs are, how atomic you need actions to be, and other similar factors. It's easier to debug problems and ensure all updates are complete when you have fewer messages to disseminate and consume. That said, not every subsystem needs the same amount of information.

You'll have to figure out what the appropriate balance is between message size (one individual message) and message volume (all the messages that make up an event).

• Don't embed files, put them in blob storage and reference them
• Think through failure states, and failure impacts
• Try to do as much as possible with the messages you use (i.e. reference data instead of embed it)
• Try to think about the impacts in terms of system resources, and debugging
• Identify the guarantees you need
  • Do you need persistence?
  • Do you need dead letter queues?
  • Are there messages that are OK to drop?

Answer 2

It really depends on what the information is for, if the intermediate state changes need to be preserved. For example, the analytics team might care in which order items were added to the cart, at what time (and which parts of the web site were visited leading to additional purchases). They probably want to send a reminder to the user if the check-out was never completed. They probably want a fast/microbatching framework that can handle a lot of events.

Whereas order fulfillment couldn't care less how the cart came to be. In fact they don't want to know about it if the cart was abandoned. They probably want a reliable framework with execute-once guarantees.