I've been doing some research on microservices, and one question I've had that is not well addressed is whether there should be multiple instances of a microservice, and if so, how to deal with this?

Suppose for example I have a system of two microservices, A and B where A depends on B. Looking at B, is B supposed to be a single instance always, and handle everything A throws at it, or is it possible to create a pool of service Bs, so they can handle the load?

If the latter, how does one manage a pool of microservice B instances, with respect to load balancing and their data ownership?


Answer mostly depends on your scalability/uptime/failover requirements. If it's necessary to have highly resilient architecture, then yes, you need to have multiple instances (sometimes even distributed across multiple data centers).

This is actually one of the benefits of microservices - you can scale different services independently - some of them are more performance critical than others so you create more instances of these. Contrast this with monolithic applications where you can scale just the whole monolith (ignoring the fact different requirements for different parts of the monolith).

If you're lucky, then you microservice is stateless - all instances are completely isolated. This is great because you can scale them almost infinitely. But often you have some state which needs to be distributed among the individual nodes. If you have relational database, you can have all nodes connect to the same DB instance, but this usually doesn't scale well. You can then setup read replicas (slaves), or even multi-master replication (read and writes possible from several DB instances which then sync), but you can still hit some limits (depending on the application and scalability requirements). Recently there's hype for distributed key-value stores - e.g. Hazelcast which can have a lot of nodes (even one per application instance) which then synchronize. This usually scales better than relation DBs, but data is usually not persistent and lost after system crash.

When you have multiple application instances, it's a good idea to hide this fact from the client - you should have only one endpoint which is served by a load balancer (e.g. HAProxy) which then directs the traffic to the actual application instances. These instances can be added/removed based on the load dynamically, the endpoint never changes.

| improve this answer | |
  • So does this means HAProxy in front of every microservice that has more than one instance? – Will I Am May 25 '15 at 17:25
  • @WillIAm yes, HAProxy should be in front of each micro service with multiple nodes. – qbd May 25 '15 at 17:41

Why not. If the microservices are stateless or have a shared database backed state then you should be able to have multiple instances of the services behind a network load balancer (F5 big-ip or Microsoft NLB as an example). I am assuming the microservices have a http api (REST). The answer is slightly more complicated if the microservices are integrated over a servicebus but even then you can have multiple instances.

| improve this answer | |
  • What I don't understand is if there is an elegant way to scale microservice instances without tying it to VM lifetimes (they are micro, so I don't always need a new VM). Also, do I use haproxy or another load balancer in front of any service with more than one instance, even those way down in the stack? – Will I Am May 25 '15 at 17:28

Services in general should be stateless, if possible. This allows you to harness one of their most powerful advantages: horizontal scalability. If you're on AWS, for example, you can set up auto-scale groups that simply create more servers when necessary and destroy them afterwards, with no loss of information or requests.

| improve this answer | |
  • Doesn't the aws auto-scaling approach only work if there is one type of microservice per instance? What if I want to add a couple more instances of a service into an existing instance that is not running at capacity? – Will I Am May 25 '15 at 17:23
  • Auto-scaling is based on instances, yes. If you are using AWS, you normally saturate a given instance before scaling up; back-filling isn't in the cards, because if you leave that capacity on the table, you're wasting money. – asthasr May 25 '15 at 18:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.