How do I avoid tightly coupling one microservice to another microservice's feature that depends on specific views of the first's data?

Question

I've seen this problem in a few different contexts now but I'm not sure what it's called or how to think about it.

Suppose I have a service, AccountService, that serves accounts from a database, e.g.

• account.id
• account.fname
• account.lname
• account.email
• account.business_address
• account.tax_id
• account.invoiced
• ...and so on.

It is a "platform" service used by many other services, such as PaymentService and PreferenceService. Suppose each of those services for some reason have to GET a list of accounts according to some criteria. For example, PaymentService needs a paginated listing of accounts with the fields business_address, tax_id, and invoiced set; meanwhile PreferenceService needs to send batches of emails to accounts whose email field has been set. (Somewhat contrived example.)

Now, I can obviously expose 2 endpoints in AccountService such as getAccountsWithValidPaymentDetails() and getAccountsWithValidEmail() that execute corresponding SQL queries. (Or more sophisticated forms of this.)

But it seems then like my AccountService is being tightly coupled to a feature specific to downstream services. That is, why does AccountService need to know what PaymentService considers a "valid" account state for billing? Or another perspective: If PaymentService modifies their definition of a valid accounts to bill, they'd have to modify AccountService—doesn't that seem wrong?

The real scenario I'm dealing with is a little more complex, sometimes joining data from multiple upstream services. And a key aspect that highlighted the problem in all the contexts I've seen is, for some reason, paginated views, i.e. getting 10 items at a time after applying some filter. In layman's terms, it's as if the fact that one must filter at the database level (i.e. in the upstream service) forces a tight coupling between the upstream and downstream service, namely by having to implement a downstream service's concept as a filter in the upstream service.

1. Is there a name for this problem or related concepts?

2. Is there a pattern that can avoid this kind of coupling?

3. If the answer is a technology like GraphQL, etc. then out of curiosity, what was the better approach before those technologies existed?

Answer 1

As is often the case, efficiency comes at the cost of having to make things efficient.

To maximize efficiency, you'd develop these separate endpoints. This ensures that you get the best query performance by letting the account service's database do the heavy lifting for you, while also keeping bandwidth down to strictly the data you need.
Note that you can still try to generalize each endpoint as much as you can/can be bothered to.

To minimize coupling and developer effort, you expose your accounts as a simple list, possibly with some basic filtering. This means that your account service is as general as it can be, but it comes at the cost of using more bandwidth that the client needs, and the client needing to do some less efficient query logic after the fact.

Which is better? Well, there are a lot of things to consider here. How many accounts are we talking about? How often is this going to be queried? How constrained are you for bandwidth/performance/developer effort? Are there privacy/security/permission concerns for the account data?

There's also the consideration that microservices might not be the right fit for you in this scenario. If you need tight coupling, you should consider different avenues.

For example, rather than an account service, you could opt for an account package (e.g. Nuget) which contains the basic DAL to access account data, meaning that it's actually your client's runtime which will access the account database; while also leaving the DAL extensible for the client to add/alter/extend its querying capability.
This also means that different services don't need to merge into one big monolith. Your client services can still be independent, but the account service is reusably absorbed into each client service. The one downside here is that updates to the account package will require a redeploy of the application, but this may be a perfectly reasonable trade-off for your scenario.

I have no real experience with the tech stack, but it also sounds like GraphQL might be right up your alley for this particular need. I can't really explain it well due to lack of experience with it. From what I know, GraphQL allows you to create an endpoint in which clients can dynamically query the data they need.

Lastly, don't forget that microservices are not inherently about runtime performance. The greatest benefit is reaped by simplifying the deployment of individual modules, which is a DevOps benefit more so than an end user benefit. Scalability of services is a great benefit too, but it doesn't always apply in every scenario.
What is most definitely not a microservice benefit is response speed. Connecting to networked endpoints is an IO task that takes significantly more time than a monolith service's in-memory runtime, and you shouldn't expect to see similar performance with a microservice architecture.

Answer 2

Microservices are a much overused pattern by people copying large companies inappropriately. In general, if you can say "I am writing two microservices" (and not "these two teams are writing two microservices") then it's not really a microservice but one piece of software that's been split unnecessarily.

It is a "platform" service used by many other services, such as PaymentService and PreferenceService. Suppose each of those services for some reason have to GET a list of accounts according to some criteria. For example, PaymentService needs a paginated listing of accounts with the fields business_address, tax_id, and invoiced set; meanwhile PreferenceService needs to send batches of emails to accounts whose email field has been set. (Somewhat contrived example.)

The classical microservice approach to that is to split the database. The payments service gets the "business_address, tax_id, and invoiced" columns and the PreferenceService gets "email" (maybe this should be the "email" service instead?). They share a key.

That results in deploying three databases, but you're doing microservices because the number of users you have and throughput of transactions doesn't fit on one database, right?

Answer 3

it seems then like my AccountService is being tightly coupled to a feature specific to downstream services.

Almost - AccountService is being tightly coupled to one or more contracts that are being used by one or more downstream services.

All libraries and products implement contracts, explicit or implicit. Contracts may be documented (using formal prose like RFCs, ideally), or embedded in code (less than ideal). -- Pieter Hintjens

The formality of an RFC is perhaps less important when the client and server are supported by the same organization. There are reported cases where the RFC formalism has been useful within a single org.

But the broad problem you face is a pretty common one - there's a new business need, and we need to figure out if that need can be met by

• Using an existing contract
• Extending an existing contract in a backwards compatible way
• Introducing a new contract

---

If the answer is a technology like GraphQL, etc. then out of curiosity, what was the better approach before those technologies existed?

General purpose query languages have been around for a long time. See, for instance, structured query language.

Notice, though, that even with general purpose messaging, we still need agreement on message semantics if accurate communication is required

{
  me {
    name
  }
}

For this sort of query to be useful, consumer and producer have to share an understanding of "me" and "name".

In other words, we haven't eliminated the need for producer and consumer to be implementing compatible contracts. We've just (a) made the contracts more flexible and (b) cleverly arranged things so that producer and consumer can each choose to leverage standardized libraries to do some of the heavy lifting (in much the same way that using a standardized media type like JSON allows us to import a parser from a library rather then rolling our own.

Answer 4

I think Andrew's question and his design was wrong at the beginning.

If he think that the AccountService need to be loose coupling with the other, then no way the business rule of validating account's payment, account's email can live in AccountService. It should be placed in PaymentService and PreferenceService. Therefore, it means that his endpoints getAccountsWithValidPaymentDetails() and getAccountsWithValidEmail() was not a good design.

PaymentService should handle the validation (business rule) in its boundary context, and cache the data in a table that contains a copy of account data with payment validated status. This cached table will need synchronized with AccountService's data (by public API or Event Sourcing protobuf). This is also meaningful in case of should there is an account is deleted (accidently or purposely) in AccountService, there still be a copy of payment transaction saved in Payment's boundary, for further viewing report.