Ways to share DTO across microservices?

Question

I am designing a system to receive data from various types of sensors, and convert and then persist it to be used by various front-end and analytics services later.

I'm trying to design every service to be as independent as possible, but I'm having some trouble. The team has decided on a DTO we would like to use. The outward-facing services (sensor data recipients) will receive the data in their own unique way, then convert it to a JSON object (the DTO) and send it off to the Message Broker. Consumers of the messages will then know exactly how to read the sensor data messages.

The problem is that I'm using the same DTO in a few different services. An update has to be implemented in multiple locations. Obviously, we've designed it in such a way that a few extra or missing fields in the DTO here and there are not much of an issue until the services have been updated, but it's still bugging me and makes me feel like I'm making a mistake. It could easily turn into a headache.

Am I going about architecting the system wrong? If not, what are some ways around this, or at least to ease my worries?

Answer 1

My advice? Do not share these DTOs among the applications in any kind of library. Or at least don't do this right now.

I know, seems very counter-intuitive. You are duplicating code, right? But this is not a business rule, so you can be more flexible.

The service that sends the DTO needs to be rigid in his message contract, like an Rest API. The service can't change the DTO in a way that could break the other services that are already consuming the information from the DTO.

When a new field is added do DTO, you only update the other services that consume this DTO if they need the new field. Otherwise, forget it. Using JSON as content-type you have the flexibility to create and send new attributes without breaks the code of the services that don't map these new fields on his actual versions of DTO.

But if this situation is really bothering you, you can follow the Rule of Three:

There are two "Rules of Three" in reuse: (a) It is three times as difficult to build reusable components as single use components, and (b) a reusable component should be tried out in three different applications before it will be sufficiently general to accept into a reuse library.

So, try to wait a bit more before sharing this DTO among the services.

Answer 2

When it comes to Microservices, services' development life cycles should be independent too.^*

Different SLDC and different dev teams

in a real MS system, there could be several teams involved in the development of the ecosystem, each of which in charge of one or more services. In turn, these teams might be located in different offices, cities, countries, plan... Perhaps, they don't even know each other, what makes sharing knowledge or code very hard (if possible). But this could be very convenient because shared code also implies a sort of sharing reasoning and something important to recall is that, whatever makes sense for a specific team, doesn't have to make it for another team. For example, given the DTO Customer, it could be different depending on the service in play, because customers are interpreted (or seen) differently from each service.

Different needs, different technologies

Isolated SLDCs also allows teams to choose the stack that best suits their needs. Imposing DTOs implemented in a specific technology limits the capacity of the teams to choose.

DTOs are neither business rules nor services contracts

What DTOs are really? Plain objects with no other goal than moving data from one side to another. Bags of getters and setters. It's not the kind of "knowledge" that worth reuse, overall because there's no knowledge at all. Their volatility also makes them bad candidates for coupling.

Contrary to what Dherik has stated, it must be possible for a service to change its DTOs without having to make other services to change at the same time. Services should be tolerant readers, tolerant writers and fail tolerant. Otherwise, they cause coupling in such a way that makes the service architecture a no sense. Once more, and contrary to Dherik's answer, if three services need exactly the very same DTOs, it's likely something went wrong during the services decomposition.

Different business, different interpretations

While there could be (and there will be) cross-cutting concepts among services, it does not mean we have to impose a canonical model to force all services to interpret them in the same way.

Case study

Say our company has three departments, Customer Service, Sales and Shipping. Say each of these releases one or more services.

Customer Service, due to its domain language, implements services around the concept of customers, where customers are persons. For instance, customers are modeled as name, last name, age, gender, email, phone, etc.

Now say, Sales and Shipping model their services according to their respective domain languages as well. In these languages, the concept customer appears too but with a subtle difference. To them, customers are not (necessarily) persons. For Sales, customers are a Document number a Credit Card and a billing address, for Shipping a full name and a shipping address too.

If we force Sales and Shipping to adopt the canonical data model of Customer Service, we are forcing them to deal with unnecessary data that could end up introducing unnecessary complexity if they have to maintain the whole representation and keep the customer data in sync with customer service.

Related links

• Microservices & Canonical model
• Avoid a Canonical Data Model

---

^{* Here is where the strengths of this architecture lays on}

Answer 3

I'm trying to design every service to be as independent as possible

You should be publishing events. Events are certain type of messages that represent a solid fact about something that has happened at a particular point in time.

Each service should have a very well defined responsibility, and should have the responsibility of publishing the events related to that responsibility.

Furthermore, you want your events to be represent business related events, not technical events. E.g. prefer OrderCancelled event to an OrderUpdated with status: "CANCELLED".

That way, when a service needs to react to a cancelled order, it just needs to listen to a that particular type of message, which is carrying only data relevant to that event. E.g. an OrderCancelled probably just needs an order_id. Whichever service that needs to react to this has already stored whatever it needs to know about the order in its own data store.

But if the service only had OrderUpdated events to listen to, then it would need to interpret on the flow of events, and it was now dependent on delivery order to conclude correctly when an order was cancelled.

In your case, however, as your are publishing sensor data, it could make sense to have a service, listen to the events and publish a new stream of "business events", e.g. TemperatureThresholdExceeded, TemperatureStabilised.

And be careful about creating too many microservices. Microservices can be a great way of encapsulating complexity, but if you don't discover suitable service boundaries, your complexity is in the service integration. And that is a nightmare to maintain.

It's better to have too few, too large services, than to have too many, too small services.

Answer 4

The most brought up reason for not sharing dtos is different bounded context could have different interpretations. However there should only be a single source of truth which should hold data that contains every field needed to serve all its consumers. I would say if you want to share dtos, understanding what is a complete DTO (from business perspective) would be a prerequisite.

Another brought up reason is that services are no longer loosely coupled anymore, as a change triggered by one service forces a change in the other. I cannot imagine what type of change could trigger this. Deletion of fields should never be done as long as there are consumers using them. New fields can be ignored if the consumer is not interested. Also keep in mind that versioning of the library provides good flexibility because each service is free to choose its version of the DTO.