How is scalability and reliability provided to microservices?

Question

Good afternoon,

I'm getting into microservices - in fact, I've managed to make few full projects based on that architecture - but they weren't really robust, so I couldn't get the full scope of using microservices.

I'm struggling with 2 concepts related to scalability and reliability, and I would appreciate a simple explanation in layman's terms.

1. Reliability Let's say that I have Docker and utilize it to run 3 instances of the same image (service). They can't all be bound to the same port, so the generated ports are usually n+1 (9000, 9001, 9002) - do I have to implement a service discovery that checks whether one of 3 ports is available for usage - or does Docker have something implemented that'll allow me to define a single port (9000) that'll redirect traffic to a service instance that is available and has the lowest load?

2. Scalability Let's say that I have 3 different services: Apples, Oranges, and Bananas. Now, let's say that Oranges are used the most and need to be scaled by adding a few instances but on a different server. How does that process work? Do I just create instances on a different server? But then what - how do they get connected to the other services?

Thanks for your time!

Answer 1

In front of your services you will typically have an API gateway, or reverse proxy. Essentially you are routing portions of a URL to the service instances. With a properly configured system, the API gateway knows about the types of services and instances. Think of it in this way:

• All your services (apples, oranges, and bananas) register themselves with a service discovery layer
• Your API Gateway looks up the instances from the discovery layer running for each of those services and load balances between them (like a simple round-robin distribution)
• Given the mapping of a URL to a service, you are guaranteed to get the right service but not guaranteed to get the same instance every request

The Spring Boot ecosystem has tools called "Eureka" for service discovery and Spring API Gateway for ... well ... the API Gateway.

Many of these functionalities can also be taken care of in a Kubernetes cluster. For example a "service" represents one of your services, and load balances traffic from a known name/port to the specific "pod" that is running your microservice. The "ingress" service in kubernetes handles the URL to service mapping.

In short, there's many ways to address the problem. If you need your system to be self-managed, then you not only need the ability to spin up and tear down instances of a service, but to route internet traffic to those instances on demand.

That's typically how the scalability works for synchronous calls from the front-end to the back-end.

Answer 2

One useful pattern is the message broker.  Though these were originally created to facilitate communication between applications, a lightweight version of a message broker can be used to ensure reliable message delivery.  (For example, you probably aren't interested in the message translation capabilities.)

Such an intermediary may queue messages, and even persist them, which allows individual services to be brought down and back up again.  This intermediary can also support a publish/subscribe notion to route messages to interested parties without them having to know about the actual endpoints.  An intermediary could do load balancing among copies of services, without other microservices having to know about replication/scalability.

Have a look at Kafka, RabbitMQ, Redis, among others.