Where to store data for Microservices Architecture?

Question

I currently build applications that are fairly monolithic. I have a one or many code bases that compile into one single binary/package and deployed on a cluster of docker containers. All of the data is stored on a single MySQL database, a redis cluster and possibly a NoSQL database for some of the data.

In this case, the bulk of my data is stored in an MariaDB RDS instance on Amazon Web Services. This works fairly well because RDS handles automated backups, and other benefits.

However, let's say that I want to split a service into its own "microservice". Where would it store its data? If I have 5 microservices, spinning up 5 RDS instances, 5 redis clusters doesn't seem to be the most cost effective and seems to be a lot of management overhead.

It seems to me that a cluster of docker containers would be more manageable for a single microservice. For example using something like docker-compose, you can spin up several docker images into as a single unit very easily. AWS has a similar concept called "Task Definitions" (to my knowledge) which you can launch on AWS Fargate or ECS. However, Fargate does not allow persistent storage, so you are basically forced to launch your database in something like RDS.

I suppose this is a fairly open ended question, and might pertain to DevOps more than actual Software engineering. How would someone design a microservice to be easily deployed on the cloud whilst being easily maintained as a separate but packaged unit of sub-services (app server, databases, cache, etc)? Using docker compose and amazon Task definitions seems to be the best way to keep consistency between development/staging/production environments, however it does have some limitations such as not having persistent storage on Fargate for example.

Just looking for examples on how someone might achieve this to help my understanding.

Answer 1

However, let's say that I want to split a service into its own "microservice". Where would it store its data? If I have 5 microservices, spinning up 5 RDS instances, 5 redis clusters doesn't seem to be the most cost effective and seems to be a lot of management overhead.

In order to a pure microservice, your services need to have autonomy over their data. Whether that's a good idea in your case or not, that's the definition. If this approach doesn't make sense for your system, you probably shouldn't be using a microservice approach.

You should really make sure you understand the goals, benefits, and downsides around microservices before embarking on trying to use them. I think this is a good reference. If you throw out the data decentralization aspect, it's not clear that you'll get much benefit from using microservices. I will probably just complicate things needlessly.