I would argue that this is more of an economics question. However, that is a judgement call that engineeres ought to be able to do. Hence, I am answering.
I am splitting my answer in four parts:
- Risk management
So, sometimes your client fails to get a response from the server. I will assume this is not because of a programatic error (otherwise the solution is to fix it, so go do that). Instead, it must be because of a fortuitous situation beyond your control...
But not beyond your knowledge. You must know:
- How often does it happen.
- What impact does it have.
For example, if failing and retrying happens only about 2% of the time, it is probably not worth to address it. If it happens about 80% of the time, well... depends...
How much time does the client has to wait? And how does that translate into costs... you see, you have a small delay in a regular application, it probably isn't a big deal. If it is significant, and you have a real time application or an online video game, this will turn users away, and you are probably better off investing in more or better servers. Otherwise, you probably can put a "loading" or "waiting for server" message. Unless, the delay is really big (in the order of tens of seconds), then it can be too much even for the regular application.
As I said above, there are more than one way to go at this problem. I will assume you already have the try-fail-retry loop implement. So, let us see...
- Put a loading message. It is cheap, helps user retention.
- Query in parallel. Can be faster, can still fail. Will require a redundant server (can be expensive), will waste server time and network traffic.
- Query in parallel to stablish the faster server and use that from there on. Can be faster, can still fail. Will require redundant server (can be expensive), will not waste as much server time and network traffic.
Now, notice I say these can still fail. If we assume that a query to a server has a 80% chance of failure, then a parallel query to two server has a 64% chance of failure. Thus, you may still have to retry.
A bonus advantage of picking the faster server and keep using it, is that the faster server is also the less likely to fail due to network problems.
Which remind me, if you can figure out why the request fail, do so. It can help you manage the situation better, even if you can't prevent the failures. For example, do you need more transfer speed on the server side?
- Deploy multiple servers across the world, and pick server by geolocation.
- Do load balancing on the server side (a dedicated machine will take all the requests, and reley them to your servers, you can have your parallelism there, or a better balance strategy).
And who said you have to do only one of these? You can put a loading message, query multiple servers that are spread across the wrold to pick the faster and only use that from there on, on failure retry on a loop, and have each one of those servers be a cluster of machines with load balancing. Why not? Well, costs...
There are four costs:
- The cost of development (usually very cheap)
- The cost of deployment (usually high)
- The cost runtime (depends of the type of application and the business model)
- The cost of failure (probably low, but not necesarily)
You have to balance them.
For instance, let us say that you earn about a dollar per satisfied user. That you have 3000 users per day. That the requests fail about 50% of the time. And that 2% of the users leave without paying when the request fail. This means that you are losing (3000 * 50% * 2%) 30 dollar per day. Now, let us say that developing the new feature will cost you 100 dollars and deploying the servers will cost you 800 dollars - and ignoring the runtime costs - this means that you would have a return of investment in ((100 + 800) / 30) 30 days. Now, you can check your budget and decide.
Do not consider these values representative of reality, I picked them for math convinience.
- Remember that I am also ignoring the details. For example, you may have little deploy cost but be paying for CPU time and you need to consider that.
- Some clients may appreciate if you do not waste their data package in redundant requests.
- Improving your product may help to bring natural advertisement.
- Do not forget opportunity costs. Should you be developing something else?
The thing is, that if you consider the problem in terms of balacing costs, you can make an estimate of the cost for the strategies you consider, and use this analysis to decide.
Intuition if foster by experience. I am not suggesting to do this kind of analysis every time. Some people do, and that is ok. I am suggesting you to have some understanding of this, and develop an intuition for it.
Futhermore, in engineering, aside from the knowledge we get from actual science, we also learn in the practice and compile guidelines of what works and what doesn't. Therefore, it is often wise to see what the state of the art is... although, sometimes you need to see outside of your area.
In this case, I would look at online video games. They have load screens, they have multiple servers, they will pick a server based on latency, and they may even allow the user to switch servers. We know that works.
I would suggest to do that instead of wasting network traffic and server time on every request, also be aware that even with redundant server, failure can happen.