I was wondering if any design gurus out there could help me understand how large web companies design their services to be highly available. The scenario I'm thinking of is:
- Client A connects to Service A
- Client A sends N requests
- On the N+1 request, Service A blows up
- Client A reconnects to Service B
- Service B services requests N+1 onwards
The only design I could think of was using a "metadata"/"discovery" service that the Client could be statically aware of. This service would give information on the best available Service, which the Client would then connect to and begin his requests, and re-query the "metadata" service when he realizes Service A went down. The application service is now highly available, but...
...the glaring problem is that the "metadata"/"discovery" service is static, will come under high load, and is not highly available, which kind of defeats the whole purpose. I suppose I could throw a lot of hardware under this service, but that's not a very good solution.
How should I go to design a real highly available web service?