I am currently facing a dilemma in the design of a message oriented distributed system following an Event Carried State Transfer pattern. This is essentially a "near real time ETL" used to populate "caches" on autonomous subscriber systems. In other words, the following principles apply:

  1. Systems are partitioned by business domains ("microservicey")
  2. There is one system of record for a message of a given type (topic), which is the publisher
  3. There can be multiple subscribers
  4. The subscribers do not control the domain of the type, and therefore do not define the business logic of the type
  5. The data is immutable on a subscriber after ingestion. Their local copy is merely cached data.
  6. A subscriber may transform the message on ingestion to fit their model of the type

For example, we might have a Widget { code, name, mass, supplierCode }. There is some single system, say, the Asset Management system, where a Widget can be created, edited, or deleted. If you were to picture this as data in a relational model, the code would be the primary key, the supplierCode might be a foreign key to a Suppliers table in that system.

Any particular subscriber may care about some or all of the elements of a Widget. Perhaps the Shipping system cares about { code, mass }, whereas the Insurance system cares about { code, name, supplierCode }. So these systems subscribe to Widget events.

Note that in this model, the Widget events are not business process events, or "domain events" defined by the Asset Management system. While I recognise that this is a popular design, I believe it is incoherent, since the downstream systems can't - by definition - understand the domain events that occur within the boundary of the publisher, since they are decoupled from that domain. The Asset Management system is simply transmitting facts. "A Widget exists with this code and these attribute values", or "the Widget with this code no longer exists", with associated metadata (message creation date time, etc) in the header. The exists flag may be in the header or in the data, it doesn't particularly matter.

Given an architecture like this (or, indeed, a "domain event" architecture), a difficult choice arises. How much of the widget should be transmitted in a single message?

My example Widget has artificially few attributes, of course. But one can imagine that the supplierCode of a Widget may be sent as a message type in itself, as merely WidgetSupplier { code, supplierCode }, leaving the Widget message containing only the remaining elements.

There seem to be major implications of this choice. To list just a subset of those, with fine grained messages we would see:

  • more messages being published, with higher transactional overhead particularly at the subscriber side
  • subscribers get greater granularity of subscription
  • Subscribers may be required to ingest partial Widgets. This can't cause domain-rules problems per se (by definition, since the subscriber cannot implement domain rules on a type it does not control). But what if, for example, a Widget has its supplierCode changed, and the Widget is then deleted? A subscriber may receive a Widget no longer exists event, followed by a WidgetSupplier event for the deleted widget, because message order is typically guaranteed within a topic, but not across topics.

With coarse grained messages largely the opposites are true - we see fewer messages, but also less targeted messages from a subscriber point of view. There are fewer issues with message order, but we can't completely eliminate cross-topic ordering problems, so the benefit of this is not "absolute".

I would like to hear from anyone else who is familiar with the decision points I've described here. Did you ever find there was one or more design considerations which essentially mandated one choice or the other for logical reasons? If there was no argument quite as strong as that, were there any pragmatic arguments that ended up being overwhelming when choosing between coarse and fine grained message data? Or are you still second guessing your choice to this day?

  • Generally speaking, decisions on message sizes of the kind you describe are arrived at by trial and error. You can certainly make educated guesses about what you want to include in each packet, but you may still have to adjust the packet size based on performance and other factors. Jul 13, 2020 at 14:56
  • @RobertHarvey So would you say that in your experience there is no overwhelming logical argument in either direction, and the problem can simply be seen as one of performance optimization?
    – allmhuran
    Jul 13, 2020 at 14:57
  • 1
    That's how I see it. Jul 13, 2020 at 14:58
  • Of course, like all software design decisions, you may choose one tradeoff over another. For example, you may trade some speed and bandwidth for simpler message handling. Jul 13, 2020 at 14:59
  • And if we're talking about the Internet, and performance is important to you, you will generally favor larger messages over smaller ones, due to the overhead imposed by each internet packet. Jul 13, 2020 at 15:02


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.