Should we share a database between two 'Microservices'?

Question

We're implementing a service that exposes data related to a particular part of a business. It will pull data in from different sources, do some ETL, and store it in Redis. It will expose this data via ReST endpoints (and possibly GraphQL).

For scalability, Redis will be replicated from London to US. The service that resides in the US will not need to do any ETL as it'll just use the existing data that's in Redis.

Because this service has two 'modes' (read and write in London, and just read in the US), we're debating whether we should split it into two services; one that writes the data, and one that reads the data.

Option one:

---

Have two services, potentially in different repos, where one does the ETL and writing of data, and one that just reads that data.

The problems that we can see with this are that we'll have two lots of everything (repos, TeamCity projects, Octopus deploys etc.). We could also end up with versioning issues as they have the potential to evolve at different speeds.

Option two:

---

One service that can either be started as 'read-write' or 'readonly'; in London, we'll start it in 'read-write' mode, and in the US, we'll start it in 'readonly' mode.

This solves the issue of multiple repos, multiple TeamCity projects etc.

The problems we can see with this approach is the added complexity of starting it in a particular 'mode'. Also, we'll be deploying code (the ETL components) that are essentially 'switched off' in readonly mode.

---

We've been reading various articles online. Some say that databases should never be shared between 'Microservices' and some saying that it's absolutely fine.

Up to this point, I've used apostrophes around the word Microservices. I've done this because so far in this question, I've been using the word to describe 'technical functionality', rather than the more well defined:

The microservice approach to division is different, splitting up into services organized around business capability

I like that description of a Microservice and I think it gives weight to my preferred option, which is option two.

But I'm no architecture expert, so I'd thought I'd ask around to see what other peoples opinions are.

What option would you choose, or would you do something entirely different?

Answer 1

The reason why it's generally preferable to separate out writes and reads is because the problems that arise from them are different, and depending on whether your application is read or write heavy you can scale both the services independently. Managing latencies during writes(which are generally higher) and connection count to your databases is much easier, a specialized service handling writes to avoid conflicts is super useful.
'Microservices' or not, the goal should be to ensure that you have enough separation between your services to allow them to scale independently and fail in isolation.
Sure, by separating out you have to keep versioning in mind, but there are so many serialising frameworks that handle versioning really really well (like Protobuf, thrift).
Separating out repos - of all the issues we developers have to deal with, the last thing should be multiple teams working on the same repo leading to a lot of conflicts.

Answer 2

It is really not easy to give a specific answer to this question without knowing the detailed requirements.

There is a recommendation: "Keep together what changes together". I would say unless you have requirements that you did not write about, start small! . Put everything in one repository. Have one build job. Deploy together. Until you have a good reason to split it up.

If the US customers will come to you for feature requests and changes completely independent from the London ones, then it could at some point make sense to split the project (code, build, deployment), but you can do this later, when the first feature request comes in.

There is nothing wrong with a 100% code reuse and splitting via configuration. Don't focus too much about the inital setup costs of deploying a "configuration" - you will probably need to configure your project anyway (e.g. on test environment vs. production environment). The question of whether to apply an ETL or not is just another configuration option.

The question whether you want to use configuration or rather implementation to split up really depends on how you expect your service to evolve over time.

Some things to think about when you architect / design your service would be: - Will the different customers always depend on the same data? - Will the customers always want the data updated at the same time (time zones)? - Will there be new customers in the future that want to have different data and different times? - Will the different customers have different non-functional requirements (e.g. performance)? If you are using Redis and replicate data between London and US you probably need a very fast access to this data? - Will there be different or the same developers for both customers? - Will the services always be released together? Do you need a downtime for the releases? How frequently do you need to release?

From architectural/design perspective you should decouple the collection of the data, transformation of data from the evaluation and transformation of the data (read). Both write and read have a transformation part, so for the London customers you do:

• Collect -> Transform -> Store and Read -> Transform -> Serve

For the US Customers you do:

• Collect -> Transform -> Store and Read -> [No Transform] -> Serve

In any case it makes sense in your software design to decouple these steps as much as possible. If your software design is good, you will have no problems to split your code later. Whether you then use "configuration" or "implementation" to make two different services is more or less a minor decision.

How you setup your codebase and your build has entire different reasons. Unless you do not have a specific reason or anticipate a problem with splitting later, don't.

Answer 3

I think you might have a common misunderstanding about what constitutes a microservice. The read and write operations are not services and a microservice is not 1-to-1 with operations. You can have a microservice with both read and write operations. That doesn't make it a monolith. This article gives a nice overview of the concepts of what is and isn't a microservice architecture:

A good way to look at an individual microservice is as a collection of related functions or endpoints composing a bounded context within your business domain.

Emphasis is mine. The question that you need to be asking is whether the read and write services are inherently coupled. This isn't as obvious as it might seem but often they are because the read operation depends on how the data is written. You might be able to avoid this by building ETL between the read side and write side but the value of doing so is questionable and it creates more complexity.

Your need to have only reads in one region sounds more like an authorization concern than an architectural one.

Answer 4

Theoretically, if two microservices read one and the same database then why do you need microservices, but not a monolith? Naturally, it comes the question, what if they read one and the same database, but every service has its own set of tables. Then it seems more acceptable to have separate microservices.

To the given question - you mentioned that Redis will be replicated, so practically you have two separate databases. Then it makes perfect sense to have two separate services, the only thing they share in common is the data model. In US you will deploy only the reader, in UK you will have a deployment of both reader and writer. Still, there is the issue that the UK reader will have the up to date data slightly faster than the US reader.