Handle Foreign Keys in microservices (microservice id from another microservice DB)

Question

I have an application made of several microservices with independent databases So let's consider 2 microservices Users and Documents The users microservice manages the users and handles everything related to them and the documents microservice contains the document info, properties, file .... Now each document belongs to a user and in the document database we have the user_id I know usually you should make a call to users to get the users data but imagine this scenario :

I have an api call that gets all the documents that are shared for a user, each document belongs to a user so if i fetch a thousand documents i have to make a thousand additional call to get the user data.

I thought of several options :

• have "common" tables like users replicated to all databases through a queue system (kafka for example) -> this will need to be maintained but gives the flexibility of having joins and foreign keys which makes it more performant
• Make an api route to take an array of ids and send back the data and handle this each time in the documents api to build back the object -> this will take additional work each time and less performance but it will be the "clean" way to work with microservices
• link each microservice to a "common" database in addition to its own database for easy access -> this is a shortcut to replicate tables but won't solve the issue and we can't join on it using ORMs

I imagine it's a common case, but I can't see anywhere what's the best practice for this use case

Answer 1

You need to analyse this from the point of view of use cases, not just "API calls".

I have an api call that gets all the documents that are shared for a user, each document belongs to a user so if i fetch a thousand documents i have to make a thousand additional call to get the user data.

Ok, but what is the use case where this API call is required?

• Is this part of a batch process which runs at night and does something with those documents? So the performance requirements are not that strict, and extra calls are not an issue? Maybe that batch process does not even need more than the user IDs?

• Is it an interactive feature for a user to check which of their documents are in use by others? Is "a thousand documents" a realistic assumption in this kind of scenario? A user who issues an API call which returns them "a thousand documents" may not have the expectation of the system returning the result immediately, and such many documents are not really easy to manage for a human in a user interface, so how many documents does a user really look at at once?

• Which user information is really required for this use case, and why/when? Maybe not not the full "user profile" information stored by the "Users" microservice, maybe only the names of those users, and maybe only when one request the details? Maybe the information can be retrieved in a background thread whilst the documents are already shown.

Of course, when all this user information is really required quickly, an array-based approach is probably the way to go, and your idea of a local user cache can be also viable. However, to make such a decision, one needs to look at the real world scenario and the real numbers, which gives the missing context to make a meaningful decision.

Answer 2

Microservices aim to be independently deployable. In this regard, it all depends how interleaved the documents and users are.

The user example is moreover a delicate example, since users may also be related to authentication and authorizations and might therefore be used in your scenario for different purposes:

• From a very general point of view, the user id is sufficient to identify the user and the ownership. The document microservice does not need to know more on the user and this is a great design fro the GDPR point of view (data minimization).

• Your document service might need to control if user id is valid (e.g if it is not obtained via a trusted JWT or SAML token). Moreover it might need to manage access rights or quotas that are specific to the document service. In this case, you may want to replicate the list of user-id from the user service and manage minimalized and specific user-id related data locally in the document service.

• In some cases, your document service might have to know a lot more on a user, because of the requirements. It may then for the sake of independence have a local understanding of what a user is to fulfill its own requirements and feed its data from the user events, with the cost of redundancy. But be careful: it’s not just a replication; if the user management service evolves, the document managment service may still work with its own view of what a user is. That’s the principle of independently deployable services.