Loadbalancing and failover in code

Question

I have HTTPS based webservices (not REST, rather old code). I am generating Java client stubs using Axis & using that to call the webservices. There are around 20 different APIs on the webservice.

I have 2 servers hosting the webservices (identical - 2 servers are being used for redundancy) and data is synced between the 2 servers - so that same API called either server produces the same result).

I do not have hardware or software load balancers or clusters.

So I am planning to implement the failover & loadbalancing in code (the failover is important, it's OK even if I don't do load balancing).

I was planning to do this in code i.e if I get a connect exception, I will trap the exception and do the same Webservice call on the other server.

I was wondering if there are any known design patterns for this or any pitfalls I should be aware of.

Answer 1

When I first read your description i wasn't sure where you planned to handle the exception, but after re-reading I am going to assume you are handling the failover in the client library. There are some JMS libraries that do something like this, so it isn't an unreasonable method. However, I would consider the following:

• is each client going to have a preferred server, and only swap if that server goes down?
• once a client picks a server that is working, will it ever switch?
• if you pick a server that works, do you stick to it?

Basically, depending on how you answer these questions I could see all of the clients getting stuck on one server if the other has small hiccup. In which case, you basically have fail-over but not load balancing.

If you swap servers randomly every request, then you have a performance problem if one server goes down.

The other big issue i can see is that if you hard code the server list into the client library then it is hard to change. It would be better to load the list from somewhere, or not at all, which is why load balancers are nice, in that they hide that list from the client completely. Assuming you want to keep the entire implementation in the client, i would make sure to make the list configurable from outside the library. As i mentioned, some JMS providers take a comma separated list of URLs for the server. You might try something like that, or take an array of URLs, or even a JSON configuration or similar.

So not totally an answer but some things to think about.

Answer 2

I am also assuming that you are talking about effectively calling the API from the client and not about the 2 servers you have. You cannot do a load balancing at the client side, because you would never know the load on each of the server, if you even look at connection time out or turn around time, you are not aware it is because of the load or because of network failures. Best is to alternatively call each servers, if at all you would like to do so.

For failover, the above suggestion holds well, where in you can inject the urls from outside. Whenever there is a connection failure, you may not need to re-try every time. instead you can also have a ping service, which pings the server every configured time and updates a common location about the status of the server and thus would help you to create and connect to the server which is available. Else you would end up making a wrong connection every time from your actual working code.