6

I have seen proponents of isolating data management and storage into microservices in a way that each microservice has its own data store (at least in a logical sense). Hence, joins and aggregations are made in the application layer.

Why is this better than using, for example, a relational database where different microservices can connect to? By doing it this way, we leverage the capabilities of the database to do joins and the likes and we can also use views to separate concerns and roles, getting the best of both worlds. If the database system is deployed with high-availability in mind, it is not a probable point of failure.

What am I missing?

  • 2
    If you want to compare things to find out which is better, you’re likely to run into some problems. Mostly that the question will invariably become “better for what?”, and even if you can nail down the requirements that might make it seem that one is better than the other, it will still be a matter of opinion. I think when people ask these sorts of questions, they just want someone to come and reinforce the righteousness of their preconceived notion. And that’s just silly. – RibaldEddie Oct 25 '17 at 4:57
  • 2
    Do you have any links/references for the proponents of isolating data management and storage into microservices? That might provide some context although this is still going to be an opinion based questions/answer – David Scholefield Oct 25 '17 at 6:56
11

A big advantage that a Microservices architecture has is that each service can be safely developed, tested and deployed independently. Having services share a database undermines this as it allows subtle changes in one service to introduce failures in other services, e.g.

  • One service might start writing data in an unexpected format into columns read by another service
  • Services might start locking records in a slightly different way, causing performance issues or even deadlocks in other services

Because of the difficulty in making sure that a given change doesn't impact another user of a database verifying changes becomes more complicated and deployments of "independent" services often become interconnected (e.g. changes need to be deployed in specific orders otherwise testing is invalidated). In addition schema changes also become substantially harder as it is difficult to identify what services could be impacted by a given change.

Having services communicate using REST APIs or message queues helps keep services independent as automated test suites can more easily verify that changes are backwards compatible, and services can safely make otherwise breaking changes without impacting other services, e.g. by introducing new API / endpoint versions.

  • I like your answer, but I would also like to take advantage of my database's implementation to hard problems like joins and aggregations. I don't like the idea of having server code reinventing the wheel above this data layer – ivarec Oct 26 '17 at 16:19
  • 1
    @ivarec you may well be splitting up your microservices incorrectly if you need to do this. Or maybe you have modularised your code excellently and still have a legitimate use case; in this scenario, utilising CQRS is probably the way forward. – Angus Goldsmith Oct 26 '17 at 16:30
2

Assume your software is large enough that several, largely independent teams work on it. You can have one database for everyone or one database per team.

One database for everyone allows you to use database features for queries, but most importantly, you can rely on database consistency because your database handles transactions. So if you have a lot of complicated transactions, this may be the better choice.

A database per team makes sure that other teams do not lock, overwrite or even destroy your data. The team defines interfaces to access and modify data and nobody can come from behind and do something in your tables. On the other hand, consistency is essentially an unsolved problem because transactions running over several team databases are hard to handle.

  • I hadn't thought about the lock issue. Good point – ivarec Oct 26 '17 at 16:20
1

At risk of mixing concerns. If all of your microservices are connecting to the one database you’ve essentially just built a more complex monolithic application with a single point of failure. For sure you can take advantage of database operations when everything is on the same server; but if the microservices data contexts are designed carefully mixing these contexts in a query should be an adhoc request rather than standard day to day usage.

You also need to think about scalability, if you have a few number of microservices you can get away with having these all look at the database server, but once you start scaling out each microservice effectively needs to work in isolation from each other.

If you have a service of type X and spin up six instances of it. Each one could have its own data store; or you use docker containers for the service itself and have this point to a common data store for services of type X.

Lots of patterns to explore

1

If you can put all your data in a single database, then in my mind you don't have the type of problem that needs microservices. Rejoice, this saves a lot of work!

Microservices are a solution for very large scale problems, when you can't have all the data in one database anymore, and/or have different teams working on different services, or have other reasons to split your software into distinct parts. That always makes things harder due to the sort of thing you mention, but it's a necessary cost at some point.

  • I'm not sure I totally agree with this - microservices can define abstraction boundaries i.e. isolate areas of code that may be changed at different rates or have different hosting/dependency requirements. For example, if you have a small area of business logic that you know may evolve radically over time then placing that in a microservice could help with managing deployment friction. – David Scholefield Oct 25 '17 at 8:26
  • I see your point, but it's also a kind of the chicken and the egg problem: if I'm not in that scale yet, maybe features from microservices like automated upscaling might help me get there – ivarec Oct 26 '17 at 16:20
0

The problem with any modular design is that you can't have modules have any awareness of the data of other modules outside of the strict contract the modules expose, because you are bound to introduce coupling on the data level you don't want. Modules should be interchangeable, but they won't be if module A depends on the data structures in module B.

I have seen an exact example of what you mention. A modular system shares the database between different modules. Joins and foreign key constraints link the tables of the different modules together, but if one of the modules has to be changed out, the coupling on the database level makes that nearly impossible. The modules can't be decoupled from the system easily and considerable work needs to be done to basically rewrite both modules, since they both depend on each other's internal data.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.