Good afternoon,

I'm getting into microservices - in fact, I've managed to make few full projects based on that architecture - but they weren't really robust, so I couldn't get the full scope of using microservices.

I'm struggling with 2 concepts related to scalability and reliability, and I would appreciate a simple explanation in layman's terms.

  1. Reliability Let's say that I have Docker and utilize it to run 3 instances of the same image (service). They can't all be bound to the same port, so the generated ports are usually n+1 (9000, 9001, 9002) - do I have to implement a service discovery that checks whether one of 3 ports is available for usage - or does Docker have something implemented that'll allow me to define a single port (9000) that'll redirect traffic to a service instance that is available and has the lowest load?

  2. Scalability Let's say that I have 3 different services: Apples, Oranges, and Bananas. Now, let's say that Oranges are used the most and need to be scaled by adding a few instances but on a different server. How does that process work? Do I just create instances on a different server? But then what - how do they get connected to the other services?

Thanks for your time!

  • 3
    Are you using load balancer? Commented Nov 3, 2020 at 3:06
  • @JamesMcLeod Like Nginx with maximum workers set? Or do you mean a programmatic one?
    – JTinkers
    Commented Nov 3, 2020 at 15:59
  • 1
    I mean any mechanism that distributes incoming requests among servers. What is the benefit of having Docker run several instances of the same server in your case over having multiple Docker hosts? Commented Nov 4, 2020 at 12:40
  • @JamesMcLeod Well, sure - I use Kubernetes and it has out-of-the-box load distributor.
    – JTinkers
    Commented Nov 4, 2020 at 18:29

2 Answers 2


In front of your services you will typically have an API gateway, or reverse proxy. Essentially you are routing portions of a URL to the service instances. With a properly configured system, the API gateway knows about the types of services and instances. Think of it in this way:

  • All your services (apples, oranges, and bananas) register themselves with a service discovery layer
  • Your API Gateway looks up the instances from the discovery layer running for each of those services and load balances between them (like a simple round-robin distribution)
  • Given the mapping of a URL to a service, you are guaranteed to get the right service but not guaranteed to get the same instance every request

The Spring Boot ecosystem has tools called "Eureka" for service discovery and Spring API Gateway for ... well ... the API Gateway.

Many of these functionalities can also be taken care of in a Kubernetes cluster. For example a "service" represents one of your services, and load balances traffic from a known name/port to the specific "pod" that is running your microservice. The "ingress" service in kubernetes handles the URL to service mapping.

In short, there's many ways to address the problem. If you need your system to be self-managed, then you not only need the ability to spin up and tear down instances of a service, but to route internet traffic to those instances on demand.

That's typically how the scalability works for synchronous calls from the front-end to the back-end.


One useful pattern is the message broker.  Though these were originally created to facilitate communication between applications, a lightweight version of a message broker can be used to ensure reliable message delivery.  (For example, you probably aren't interested in the message translation capabilities.)

Such an intermediary may queue messages, and even persist them, which allows individual services to be brought down and back up again.  This intermediary can also support a publish/subscribe notion to route messages to interested parties without them having to know about the actual endpoints.  An intermediary could do load balancing among copies of services, without other microservices having to know about replication/scalability.

Have a look at Kafka, RabbitMQ, Redis, among others.

  • Oh, I do use RabbitMQ for cross-service communication, but the issue is - how is the communication implemented between different servers?
    – JTinkers
    Commented Nov 2, 2020 at 18:25
  • @JCode With RabbitMQ it's implemented in exactly the same way as it would be for just a single server. Messaging is all routed via the broker, there is no direct communication between servers Commented Nov 2, 2020 at 20:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.