I'm developing a microservice system. I have a discovery service (Eureka) for all my services and an API Gateway (Zuul) to call each service via an URL.

My problem: How can I assure that a service call via my API Gateway is high available?

If the gateway dies, my services aren't callable anymore. I could start multiple gateways but then the client needs to know which gateway to call. Wll than I need a load balancer between the client and my gateway. But what happens if the load balancer dies?

I could set up a second system and route a DNS entry to both systems but still, a broken load balancer could break one complete system?

So, is there a good solution to make load balancers or a api gateway high available? Or will there ever be a single point of failure per system?

  • On the one hand. making things reliable is not the same than making them available. If you need high availability you have to deploy at least 3 replicas of the Eureka (the more the better), as many Zuuls and services as your SLA requires. On the other hand, making thing reliable is a matter of implementing and configuring things well. So, what's your question? Making things reliable or available?
    – Laiv
    May 9, 2019 at 14:46
  • Sorry, meant availability. But my clients do only know one Zuul URL, so Zuul is always a single point of failure, isn't it? Or the load balancer in front of Zuul is.
    – Milgo
    May 10, 2019 at 5:40
  • You can have as many layers as you need between the Zuul and the API clients. But yes, there should be a good balancer in front of all of that. In front of the load balancer, there could be something else, for example, an API Manager which usually also acts as API Gateway (among other things).
    – Laiv
    May 10, 2019 at 6:22
  • And why is an API manager not a single point of failure?
    – Milgo
    May 10, 2019 at 7:58
  • 1
    VTC; This is really more a system and network engineering problem than it is about software and is a well-solved problem.
    – Blrfl
    May 10, 2019 at 20:13

2 Answers 2


I would have thought this would have an answer by now as its a standard problem.

I'm going to have to give a fairly generic answer im afriad as its not my specialty.

Basically you have two load balancers, a master and a slave and a way to switch between them if one fails. Usually a shared ip address which is either active or not.

Now you have both boxes talk to each other so they can detect when one fails.

If the master fails, say its unplugged, the slave detects the lost connection and activates, becoming master. Usually by grabbing the network traffic for the shared IP address.

You can have whole sets of machines in such a cluster, as long as they all talk amongst themselves and decide who is the master you are good.


Rather than reinvent the wheel I would explore software as well as potentially PaaS offerings in various cloud providers that solve all the sticky problems of HA and dealing with single points of failure.

There are hardware load balancers (eg. F5 Big IP), software load balancers (eg. ngnix, HAProxy, Kubernetes, etc...) and cloud load balancers (AWS Elastic Load Balancer) and all of these have redundancy and various failover schemes to prevent single point of failure.

In terms of DNS records, there are various strategies that employ this as well to ensure that redundancy and failover can be handled smoothly. Scripting the update of DNS records, adjusting TTL on DNS entries, utilizing a CDN like Akamai for automatic DNS resolution and geolocated endpoints. Many cloud providers have platform features that you can leverage here as well.

The possibilities are legion and one can write many books on the subject as a whole, so it is important to do your research and find out what makes sense for your application architecture.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.