Granularity of API Gateways to Service Balancers

Question

OSS API Gateways typically come bundled with built-in load-balancing capabilities. We see this with Netflix Zuul and Kong.

In a service-oriented architecture, where many microservices are talking to many others, is the idea that each service has its own API Gateway, with its own load balancer that can serve traffic to all the nodes of that service? Or is it typical to have a single Gateway routing traffic to load-balancers for each service behind it? And if the latter, then why do OSS gateways typically feature load-balancing as first-class citizens?

For instance, pretend a backend consists of 3 web services: Orders, Payments & Shipping:

• Orders keeps track of catalog (orderable) items and receives orders from client apps
• When Orders WS receives an order request, it contacts Payments WS to handle the payment
• Then after Payments returns a successful response (payment was successful), Orders WS reaches out to Shipping WS to handle warehouse/packaging/shipping logistics

Is there typically:

1. A single API Gateway for the client app to call into, which then routes the order request to Order WS's load balancer (say, at https://orders.example.com) which in turns serves traffic to one of the Order WS instances (kicking off the order processing flow); or...
2. One API Gateway for each service? And the client app calls Order WS's Gateway/Balancer, which then forward traffic to an Order WS instance, which then calls Payment WS's Gateway/Balancer, which then forwards traffic to a Payment WS instance, etc.?

I was under the impression it was the former case, but Zuul and Kong both advertising load-balancing as a major feature is throwing me off. Any ideas?

Answer 1

There's nothing stopping a single API Gateway instance being used for all services, while still managing inter-service communications. Each individual service will simply have to send all requests to other services through the central API gateway. For instance, the following sequence diagram demonstrates the path through the network of your order webservice example:

In order to reap the benefits of an API gateway, all communications to services—both inter-service and external—must occur through the gateway (or gateways). The gateway adds value as a sort of middleware, including features like request tracing, surgical routing. If requests are sent directly to the underlying services, the observability and features of the gateway are entirely lost.

Load balancing doesn't actually change anything here. In the single API gateway model, requests to a service through the gateway are distributed among all backend servers for the service. Practically, the load balancing performed by the gateway will look exactly like a DNS round-robin-based load balancer if the gateway didn't exist at all. However, the gateway can provide more intelligent load-balancing (monitoring service server health, etc) than simple DNS-RR load balancing can.

---

That being said, Netflix internally runs an instance of Zuul for each service. See their architecture:

Source: How We Use Zuul At Netflix

This still maintains much of the benefits of an API gateway, however it has one obvious problem—it passes the problem of service discovery on to the service consumers (other services and/or clients):

In the case of a single API gateway, the gateway is reachable at a well-known IP address or DNS name, and all service discovery can occur within it. This means that code calling some service will only need the name of that service and the well-known address of the single gateway.

However, with one gateway per service, code calling some service will need to know the address of that service's API gateway. This would require an external service discovery system, perhaps something like Netflix's Eureka.

---

Hope this helps! I've spent a good amount of time researching this, so I'm confident in this information.