0

Let’s say I have a booking microservice (single instance) with its own DB.

If I want to horizontally scale the service with multiple instances(exact same copy of the business logic code) running on multiple servers(with zookeeper keeping a track of service registry and centralized configuration)

What efficient changes can be done on the DB side? The first thought that comes to mind is maintaining separate DB instance for each instance (since they are running on different servers)

But I want to be careful about the integrity of DB writes, and uniformity of DB reads - basically consistency in data reads and writes.

In my case of booking service, how to ensure that once a certain booking is done with a booking id, the other instances are immediately aware and don’t repeat it.

The below article does scratch the surface on having a cache maintained by each instance on Apache Kafka, and then maintaining datastore for each instance - Basically an event driven mechanism. Is the understanding correct ? In that case why would the cache be needed ?

https://stackoverflow.com/questions/33399988/microservices-datasource-per-instance-or-per-microservice

If I go with one datastore per instance, the most prudent question I think would be what synchronization mechanism to use to keep all these datastores consistent with the data, as they should have exactly same data, and if a read happens, how to make sure it happens from an instance with the latest data. If I am delegating requests using a loadbalancer, how does it know which instance already has latest data in DB, hence the read request needs to be forwarded to it ?

Basically trying to bring it all together in my system design plan.

Need some expert guidance on this, please.

5
  • Why are you scaling to multiple instances? Dec 3, 2023 at 21:22
  • 1
    @PhilipKendall what if my original single instance can’t handle all requests, and the number of users keep increasing? Isn’t horizontal scaling an advantage microservice has over monolith ?
    – dig_123
    Dec 4, 2023 at 4:29
  • 1
    Well, that would depend why your single instance can't handle the requests. If you're constrained by your upstream's network bandwidth, adding more instances makes no difference. This site works best when questions are asked about actual problems you face, not hypotheticals. Dec 4, 2023 at 8:04
  • 3
    Microservices aren't innately easier to scale than monoliths. Their advantage is that you can scale specific services instead of the whole thing - but those specific services still need to be designed for it. Dec 5, 2023 at 15:12
  • @JacobRaihle I'd say it depends a bit on your micro service "definition", for many that term now includes a typical microservice setup, like running on a k8s cluster. And if you compare that with a classic dedicated/virtual server setup meant to handle a big monolith it would seem more easy to ad-hoc scale up a service horizontally as it is an innate feature of the infrastructure. (In the more strict sense you are right of course, I'd just wanted to point out that OP might use the term in its more broader form...) And of course, you might just shift the problem to the one running the cluster. Dec 22, 2023 at 20:31

2 Answers 2

4

What efficient changes can be done on the DB side? The first thought that comes to mind is maintaining separate DB instance for each instance (since they are running on different servers)

Absolutely do not do this.

You may already have a failover replication system for your DB. That often makes it possible to handle reads from the replica. The most practical solution is to simply have two very large powerful DB machines; that can scale an incredibly long way, because the DB consumes less CPU than the services. Stackoverflow itself uses this architecture.

Once your company gets to the size where that no longer works, you can spend a few million dollars on consulting with the experts to migrate to a bigger solution. But first you have to get to the size of FB/Twitter.

2

Deal with potential problems in a logical order, avoiding the temptation to prematurely optimise.

  • Write your microservice to be scalable. The most important part of this is that it is stateless. You should be able to spin up / shut down individual instances of the service without issue.

  • If you ever need to scale up your service due to demand, your code is not a blocker, and it should be relatively straight forward.

  • If you need shared state between service instances, use a shared cache, such as Redis.

  • Scaling your database is a different issue. It makes sense in 90% of cases to have a single database that all copies of your service to connect to, until your load causes problems.

  • Horizontally scaling databases do exist (or correctly utilising a no-SQL db such as Cosmos, if you can properly design around the partition keys), but they tend to be more complex and expensive.

  • Other options do exist if you reach this point. E.g. would you start to have a database per tenant, per service, instead of a single database for all?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.