Why is it considered a good practice to isolate data management in a microservice?

Question

I have seen proponents of isolating data management and storage into microservices in a way that each microservice has its own data store (at least in a logical sense). Hence, joins and aggregations are made in the application layer.

Why is this better than using, for example, a relational database where different microservices can connect to? By doing it this way, we leverage the capabilities of the database to do joins and the likes and we can also use views to separate concerns and roles, getting the best of both worlds. If the database system is deployed with high-availability in mind, it is not a probable point of failure.

What am I missing?

Answer 1

A big advantage that a Microservices architecture has is that each service can be safely developed, tested and deployed independently. Having services share a database undermines this as it allows subtle changes in one service to introduce failures in other services, e.g.

• One service might start writing data in an unexpected format into columns read by another service
• Services might start locking records in a slightly different way, causing performance issues or even deadlocks in other services

Because of the difficulty in making sure that a given change doesn't impact another user of a database verifying changes becomes more complicated and deployments of "independent" services often become interconnected (e.g. changes need to be deployed in specific orders otherwise testing is invalidated). In addition schema changes also become substantially harder as it is difficult to identify what services could be impacted by a given change.

Having services communicate using REST APIs or message queues helps keep services independent as automated test suites can more easily verify that changes are backwards compatible, and services can safely make otherwise breaking changes without impacting other services, e.g. by introducing new API / endpoint versions.

Answer 2

Assume your software is large enough that several, largely independent teams work on it. You can have one database for everyone or one database per team.

One database for everyone allows you to use database features for queries, but most importantly, you can rely on database consistency because your database handles transactions. So if you have a lot of complicated transactions, this may be the better choice.

A database per team makes sure that other teams do not lock, overwrite or even destroy your data. The team defines interfaces to access and modify data and nobody can come from behind and do something in your tables. On the other hand, consistency is essentially an unsolved problem because transactions running over several team databases are hard to handle.

Answer 3

At risk of mixing concerns. If all of your microservices are connecting to the one database you’ve essentially just built a more complex monolithic application with a single point of failure. For sure you can take advantage of database operations when everything is on the same server; but if the microservices data contexts are designed carefully mixing these contexts in a query should be an adhoc request rather than standard day to day usage.

You also need to think about scalability, if you have a few number of microservices you can get away with having these all look at the database server, but once you start scaling out each microservice effectively needs to work in isolation from each other.

If you have a service of type X and spin up six instances of it. Each one could have its own data store; or you use docker containers for the service itself and have this point to a common data store for services of type X.

Lots of patterns to explore

Answer 4

If you can put all your data in a single database, then in my mind you don't have the type of problem that needs microservices. Rejoice, this saves a lot of work!

Microservices are a solution for very large scale problems, when you can't have all the data in one database anymore, and/or have different teams working on different services, or have other reasons to split your software into distinct parts. That always makes things harder due to the sort of thing you mention, but it's a necessary cost at some point.

Answer 5

The problem with any modular design is that you can't have modules have any awareness of the data of other modules outside of the strict contract the modules expose, because you are bound to introduce coupling on the data level you don't want. Modules should be interchangeable, but they won't be if module A depends on the data structures in module B.

I have seen an exact example of what you mention. A modular system shares the database between different modules. Joins and foreign key constraints link the tables of the different modules together, but if one of the modules has to be changed out, the coupling on the database level makes that nearly impossible. The modules can't be decoupled from the system easily and considerable work needs to be done to basically rewrite both modules, since they both depend on each other's internal data.