I have a web app that searches across 2 APIs right now. I have my own Restful web service that I call, and it does all the work on the backend to asynchronously call the 2 APIs and concatenate them into one result set for my web app to use.

I want to scale this out and add as many other APIs as I can (currently looking at about 10 more). But as I add APIs, the call to my service gets (potentially) slower and more complex. How do I handle one API not responding ... and other issues that arise?

What would be the best way to approach this? Should I create a service call for each API, that way each one is independent and not coupled to all the other calls? Is there a way on the backend to handle the multiple API calls without all the extra complexity it adds?

If I go the route of a service call per API, now my client code gets more complex (and I have a lot of clients)? And it's more work for the client, and since I have mobile apps, it will cost the client more data usage.

If I go one service call, is there a way to set up some sort of connection so I can return data as I get it, in case one service call hangs?

  • 1
    Create a proxy server to handle all the API calls and hide all the complexity inside. So you will handle every issue in one place and not with every API call in the application.
    – superM
    Mar 1, 2013 at 14:50
  • @superM - I have that now ... but do I separate each API call or group them all as 1 on the proxy server?
    – Martin
    Mar 1, 2013 at 15:07
  • I would group them all in one proxy server, using namespaces or something similar for the inner structure of the proxy server.
    – superM
    Mar 1, 2013 at 15:09
  • The handling strategy for slow response would depend on whether the results are cacheable.
    – rwong
    Jan 20, 2014 at 3:21

3 Answers 3


In systems like this, you always have to consider that as far as your code is powerful, at least a call will be delayed, and if your system is successful, that event will happen hundreds of times making your system slow, then maybe unstable.

I do not know the nature of the integration, maybe game providers, maybe social networks or anything else. But if your application is not academic, and it represents a real business requirements (in production) I think this is part of the things to do:

Before code

  1. if possible, all systems that are integrated, they must be aware of the timeouts that are demanded from you (anticipating this from a contractual point of view, it is a good idea mostly because no one guarantees 100% of up-time. It's myth).
  2. if point 1 is not possible, be objective and consider what we are willing to make our customers wait in front of the monitor (45 seconds, 10, 5, 2?). Found the value, we know how to set up the rest.
  3. Should be considered a constant calls/sub-calls prifile and logging;
  4. Should be planned for the possibility of suspending one or more services from the configuration (and not from code);


As for the technique, I think it is convenient to create a semaphore.

N functions will be called, the callback of each sub-function will be considered only if time set for each integration are respected.


  1. A call is made to the main function,
  2. This considers what services are enabled to be contacted,
  3. The callback functions are configured,
  4. The sub-functions are called,
  5. While making calls, and while the general timer has not expired, all callbacks are considered to be unified in the response of the main function.

You need to look into threads and/or coroutines (Sometimes called fibers, goroutines, gevents, in different languages).

When a request comes in, you spawn off 10 threads/coroutines, then wait for them all to respond, or for a timeout. To the "web server", your request is simply sleeping for a few seconds. But other threads/coroutines are actively doing work. If the work isn't done in N seconds, the threads/coroutines are killed off and only the results you have are returned.

Example in Go.

Threads are heavy weight (harder to scale), but generally better supported depending on the language. Coroutines are more efficient (less RAM, less CPU), but have problems on some languages/libraries.


How would you handle one of the API calls failing? If only some of the calls succeed is it still a valid response to the client?

Since websockets aren't widely supported and you are talking mobile platforms I think you have limited choices.

Long running requests

If the expected response is such that it may be processed in chunks before all requests complete, you may be able to stream the results in a single connection. Ajax patterns provides some good information and examples for reading from a response stream before it has completed.

Using this approach, you would make a single request and handle the response as each API request completes.

Along with this, you will want to timeout individual API requests aggressively so that a hung service doesn't heavily impact your interface.

Yo don't want the client to timeout before done.

You could go as far as skipping calls for an expanding amount of time for any services which timeout or fail.

If it's an aggregated news feed or something this may make sense but you could poll and cache the feeds server side instead.

Long polling

Instead of a single request, you could make an initial request and then poll for additional data. This has the advantage of not holding the connection and appearance of being a push.

This would completely defeat your intent of limiting the number of connections.

In conclusion

With all this said, I still think you need to consider all the factors of your design.

  • what is the upper limit of API calls that will be made behind this proxy? 10 may be ok but at 20 or 30 is this still the right design?
  • do all clients need the data from all services? Are you saving a little at the expense of all?
  • what is the quantifiable amount of bandwidth and processing time saved using the proxy for say 1000 requests? Is this worth it?
  • Didn't realize this question was old... Oh well
    – crad
    Jan 20, 2014 at 7:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.