7

TLDR:

If microservice A needs to combine data from services B & C (that are not aware of one another), should B (or C) do the data merging for service A, or it will be up to service A?

In Details

So our retail website has a microservice that is responsible for Products data:

Get products by category, or do a GET for a product ID and get info about the product's (image, price....).

The websites talks to this service to build all kind of "Product Lists" pages for the users - e.g. "Sports Products".

We have a new feature on the website - Hot products, were the website will display a nice list of Hot Products for the user. Imagine a ticker with Hot Products on the checkout page.

So we have a new microservice that is responsible for the business logic of finding what are the relevant products for this specific user and return them to the caller (e.g. Hot products -> productIds: 456, 789).

The business logic of the Hot Products service is less important. We can even imagine that for phase 1 it is: if "morning" then products 123 & 456. If "noon" then products 789 & 3030... and for phase 10 it will be an AI algorithm.

On the website, we need to display more data about the product (name, image, price,....). This data is can be found in the products microservice.

And here comes the question:

Who is responsible to translate the Hot products's meager IDs recommendations into something that the Website can use?

Option 1 - The Hot products microservice will also take care of that. Instead of returning only a list of IDs, it will also call the Products service in order to bring all the data that is needed by the UI and return a much richer response to the caller.

Drawbacks of option 1 - involving the display/usage business logic of "what needs to be done with the hot products list" into the business logic of the Hot Products microservice. If another caller needs different product information (e.g. brand) the Hot products microservice needs to be changed or at least "know" about it

Option 2 - The Hot products will keep on returning only the IDs of the products, and it is up to the caller's responsibility to do with the data whatever it needs - e.g. calling the Products microservice to bring the data it needs about the different products.

Drawbacks of option 2 - more work on the caller side. No more "one-stop-shop" at the Hot Products microservice.

So where do we draw the line of responsibility or "Domain Driven Development" in such a case?

  • How does the hot products service get product ids? – Rik D Oct 10 at 8:55
  • @RikD another process is streaming data into it's DB. – riorio Oct 10 at 9:06
  • Can you tell a bit more about the hot products service? How does it determine which products are hot? Btw I think you switched the drawbacks. – Rik D Oct 10 at 11:16
  • 2
    How are Hot Products determined? Or more to the point, why are Products and Hot Products 2 separate micro-services? To me it looks like you’ve defined an entire service to satisfy a single query. I don’t like either option until I am satisfied that all of this extra ceremony/infrastructure/communication (complexity!) is worth it. – king-side-slide Oct 10 at 13:39
  • 1
    You should probably start by adding this as a DLL (or whatever is the equivalent in the technology you're using) to the Products service itself. That way, you can take some time to figure out whether you want HotProducts to be more of a vertical slice (closer to your Option 1), or a specialized component that deals exclusively with the hot products logic (closer to Option 2). Once you have the boundaries and the interfaces figured out, than you can extract it (if at all necessary) into a separate service, and you'll know exactly who calls whom, and what data to send. – Filip Milovanović Oct 10 at 14:19
4

It is pretty common for one microservice to need data from another microservice to service their consumers (in the form of API responses). Fetching data synchronously couples the two microservices tightly. Leaving it to the consumer to fetch data from another Microservice can lead to chatty APIs and slow response time.

A few conventions to confirm your domain model and push it further to arrive at the solution you are looking for:

Bounded Contexts make good Microservice boundaries.

One Bounded Context (BC) can contain more than on microservice, but one microservice should never span across BCs. A domain concept makes sense only when considered within a BC. It may mean something else in a different BC.

Your boundaries - Product Management and "Recommendation" Engine- seem to be correct, IMHO.

The concept of Product may mean one thing in the Product Management Microservice but can be subtly different in the Recommendation Microservice. The differences can be structural or behavioral.

Microservices share nothing.

All data and APIs related to the Microservice are enclosed within it. If another Microservice needs this data, they are exposed as well-defined services (RPC-based communication, for example) or as APIs (REST-based, for example). Accessing another microservice's data via the database is strictly forbidden.

Microservices are connected over a common message channel.

Data points that are related to multiple microservices are published on a common channel as Events. Interested Microservices have subscribers watching out for the event, pick it, and process it for internal use. In DDD parlance, these are Domain Events.

An Aggregate in one Microservice could be a Value Object in another.

Product Management BC is the owner of Product-related data. Other microservices may retain/cache portions of that data within their boundaries (like you are doing with Product IDs, in your case).

Read models can be used to serve APIs with different needs.

You can populate a read model with data prepped and ready to be served in API responses. In your example, you would have a row (or multiple rows) per user in the read model with ready-to-ship data in the Recommendation Microservice.

There can be more than one read model per data structure, as dictated/required by API responses.

It's perfectly valid to construct and store data in different formats to cater to different APIs. You would use Domain Events with a pub-sub model to populate these read-only data structures in the background.

An API request should be handled in entirety by one single Microservice.

Unless you are using reactive architectures and you can gather data from microservices in parallel, you are better off dealing with the request within one single Microservice in entirety.


So there is a third option of storing a copy of Product data (only what you need) as part of the Recommendation Microservice and using it when constructing the response for Hot Products.

The Product data here is treated as a cache, populated in the background (typically by listening to events being published from the Product Management Microservice), and should be reconstructable in entirety. Most importantly, the Recommendation engine should treat this data as read-only, and not add any additional metadata into it.

| improve this answer | |
2

Domain Driven Design has Bounded Contexts. Areas within a system that share the same Ubiquitous Language. Words within the context have a specific meaning, which might be (slightly) different outside the context.

Within a Bounded Context, one or more Aggregates exist. Graphs of related objects (Entities and Value Objects) which encapsulate the business rules.

When defining the boundaries of microservices, some people prefer a service per Bounded Contexts and others prefer a service per aggregate root. It will depend on several things, like the number of available teams and the required scalability. A benefit of a service per Bounded Context is that all knowledge of the problem domain is handled by a single team. If the choice is made for a service per Aggregate and there are multiple teams working on different services, a lot of communication is needed between the teams. Because all the services operate within the same Bounded Context, it’s important all teams use the same Ubiquitous Language.

Most applications benefit from fast reads. Users don’t like to wait for a page to load. Writes can be a bit slower; when a user clicks a button they are much more forgiving if it takes a little while. DDD aggregates often load a lot of data to make the required business decisions. Because of this, it’s best to use the DDD model for writes and use a separate (page specific) read model which does not contain logic, for fast reads.

With all of the above in mind, let’s look at the question.

Products and Hot Products most likely belong to the same Bounded Context, assuming they share the Ubiquitous Language.

Therefor it makes sense to combine these features into a single microservice. This will obviously immediately solve the problem, because the API can return Hot Products instead of Hot Product Ids.

We want to make queries fast, so we need a mechanism to determine which products are hot. Let’s assume products which are often bought together with each other are hot. When someone buys more than 1 product, we update our hot items. Now when we want to display hot items for some product, we can simply query the hot items system for products that are often bought together with the current product. No business logic involved in the query.

If there are good reasons why the Hot Products service cannot be integrated into the Product service, the next logical step would be to integrate at the database level. This conflicts with the principle that each microservice should have it’s own database, but I think it makes sense when both services operate within the same Bounded Context. And let’s be honest, if the Products API is just an HTTP interface for a SQL server, we might just as well query the database directly. Both of the solutions you suggested yourself also have a strong coupling between the two services.

Finally, if you want to keep your current architecture, I would go for your first option. It’s the client that dictates the contract of an API. The client needs Products, not Ids. A service has to make things easier for a client, not harder. Give the client what it needs; return products.

| improve this answer | |
1

Option 3: Go RESTful

You already have a product microservice. Make the microservice return the HTML. I.e. make it capable of displaying products.

This would be a huge win, because you no longer need to return "data" and have everybody understand what individual pieces mean, how to display them correctly, etc, you can just link to different products, product listings.

This also makes IDs obsolete. You don't return IDs, you return URIs. I guess URIs are also IDs, but you get the point.

You can make hot products a different service if you want. The result of which is a link or even a redirect straight to the product microservice for the display.

It's simple and fast, because data is not exchanged, loosely coupled, because services only know URIs, they're essentially integrated through the UI, so they don't really call each other directly. You don't need a message bus, etc.

There is some more info about this here: Self-Contained Systems.

| improve this answer | |
0

TLDR;

From a pragmatic perspective, you may be prematurely spawning a service. But, if you decide you need a new service, the service should provide "essential" value on its own.

Depending on the complexity of the logic, it may be enough of a value-add just to return a list of ID's. Maybe not. In my world, if you've justified spawning a new service, you should be able to justify a full team owning it, and that team should be operating like a small business. The consumers are their customers. Negotiate with your customers.

Pragmatically speaking ...

There are three reasons1 to spin off a new service, be it micro or otherwise. They're not binary reasons. So, you still need to use some judgement, but generally speaking, you want think about spawning a new service if:

  1. The interface you're vending will be consumed by multiple other services (or teams).
    This is as close to a binary indicator as it gets (even though it's not). When N other services or teams are consuming a feature or interface, the interface needs to scale differently, and the team needs to balance business priorities/requirements independently from any interface not being consumed by exactly those same N consumers. If N > 2, I'd almost call this a binary choice: Spin off a separate team to manage this as a service.

  2. You need to and are able to confine performance degradations and failures around an interface.
    If getting the list of "Hot Products" is computationally expensive or buggy and is not an essential aspect of "Products", in your example, it can potentially warrant being its own service. You can protect the more core features from those performance issues with try-catches and timeouts around the service calls, which should be running, failing, and throwing a fit on isolated, independent infrastructure.

  3. For any reason, a full, independent team can build and/or operate the "interface".
    If the business logic is complicated enough for 3 to 8 people to be dedicated to it, it's a signal that the team and service should be independent. Or, if for any other reason the feature/interface has a dedicated team, subteam, or "cluster" of people working on it exclusively, officially or otherwise, it's a signal to bust that service/team out into its own independent thing.

Apart from those signals... Don't hunt for opportunities to spin up new services where you don't need them. If a feature or interface fits inside an existing service, put it there. If you have your suspicions that the feature/interface might need to be a separate service, code against the interface as though it were a separate service ... if the time comes to form a new team around it, the original consuming service will be easy to update, and the feature will by easy move out.

The reasons you spawn a new service to begin with signal how to define boundaries. (Again, pragmatically speaking.) Each service should be owned and operated as an independent micro-business. The owner/operator should operate it as such. The consumers of the service are your customers. It's each service owner's job to stay in business ... Negotiate your service boundaries and interfaces accordingly.

1. There is one notable caveat. My "three signals" assume you're agonizing over the decision to spin off a related and relevant feature or interface into a new service. If the thing you're wondering about is literally unrelated to any existing service, you have my permission to make it its own service ...

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.