# API Class with intensive network requests

I'm working an API which works as "intermediary" between a REST API and the developer.

In this way, when the programmer do something like this:

User user = client.getUser(nickname);


it will execute a network request to download from the service the data about the user and then the programmer can use the data by doing things like

user.getLocation();
user.getDisplayName();


and so on.

Now there are some methods like getFollowers() which execute another network request and i could do it in two ways:

1. Download all the data in the getUser method (and not only the most important) but in this way the request time could be very long since it should execute the request to various urls

2. Download the data when the user calls the method, it looks like the best way and to improve it i could cache the result so the next call to getFollowers returns immediately with the data already download instead of execute again the request.

What is the best way? And i should let methods like getUser and getFollowers stop the code execution until the data is ready or i should implement a callback so when the data is ready the callback gets fired? (this looks like Javascript)

• It is hard to answer this quesiton with out knowing "Why you need an intermediary between the Rest and the consumer" and "Is the consumer always going to need the followers, and how big is that data set going to be?". Generally the rest service is already an intermediary between the data and the consumer of that date. Why would you want another? – SoylentGray Aug 18 '14 at 16:17
• To avoid duplicate work in the application, i mean: i would need to execute the request, parse the data and work with it everytime i need something from the service. Then do the same thing in another part of the app. I think create another intermediary would improve a lot the code and it will be more reusable if i need it for another application i can just use the same API and everything already works – Marco Acierno Aug 18 '14 at 16:25
• About the size of the data: it would be big and small. It depends. That's why i don't know how to proceed. I could rewrite the second part of the question with "I should care to download the data in another thread or let the developer handle that?" – Marco Acierno Aug 18 '14 at 16:26
• So this would be an API that is internal to the application that manages the calls to the service? Basically you are creating the data layer for this application or at least for this functionality? – SoylentGray Aug 18 '14 at 16:49
• Yes, it will execute the request, parse the request, put in the object and return the object with the data – Marco Acierno Aug 18 '14 at 16:50

It is more convenient for you if you grab all the relevant data (about the user, the followers, etc.) in just one routine. You don't have to worry about the partitioning of the access among several calls, or having some information available and some not, etc.

But having one big, honking call that takes a (possibly) long time to execute, and consumes a lot of both local and remote server resources to boot--that is a low scalability, anti-social design. I'm not saying never, ever do that--but it's a lot more appropriate in exploratory prototypes than production code.

If you wish your User object to mask interactions with the REST API, then your getFollowers method should invoke the required behind-the-scenes shuffle to grab the followers information, then cache it locally. That is completely appropriate and typical. While the first request will take much longer than subsequent requests, c'est la vie.

Unless you are in a highly performance-constrained or latency-sensitive situation, it is unlikely that you want to involve asynchronous request/return mechanisms. They are complicated to get right, and notably complicate both your logic and that of User consumers.

You see async calls often in JavaScript--along with an army of callback handlers, registered event handlers, promises, futures, and other rendezvous techniques. That's because JS has no other good mechanisms for handling threading/concurrency, and async has become the language idiom. You can do async calls in Java, for example with Futures. But by and large, Java programs tend to favor threads over async calls, and Java API consumers predominantly expect synchronous methods. Unless you have a specific requirement to do things async, or believe doing so will yield a substantial performance/latency win, don't. Don't do things that require your consumers to adopt an unexpected model of concurrency.

Note also that many REST APIs are paged. Requests like "get followers" or "get posts" return the first 20, 50, or whatever. Getting the next N objects requires a subsequent call. In those cases, making network / REST APIs calls is something that your getter methods might have to do across the life of your User objects, not just once or twice "up front."

• I don't get the It is more convenient for you if you grab all the relevant data (about the user, the followers, etc.) in just one routine. You don't have to worry about the partitioning of the access among several calls, or having some information available and some not, etc. part. About the async call, i was just stuck at the part "Other programmers except to see this call async?" and i should admit that my answer was "no" most of the time because it could means that they should change their multi-thread design and it could make the code more complex without a particular reason. – Marco Acierno Aug 18 '14 at 16:38
• About the fact that most REST API are paged, yes i'm aware of this. But i was just going to provide something lke getFollowers(page) and use cache if i have it/download if not – Marco Acierno Aug 18 '14 at 16:39
• By "more convenient" I mean: You can just grab all data in one method. You don't have to worry about having the base user data, but not the follower data, and then having to separately fetch follower data (and cache it) with getFollowers (or getPosts) kind of calls. – Jonathan Eunice Aug 18 '14 at 20:01
• The fact that stop me to do this is i don't think it's natural to have a method that the first time require 20 seconds to execute and then only some milliseconds. Even if documentated and even if i provide a method like hasFollowersReady it doesn't look so intuitive. It looks random. – Marco Acierno Aug 18 '14 at 23:07
• Fair point. Users dislike unexpected and variable delays more than most other things. However, remote procedure calls and network APIs are infamous for very long runs. Obviously try to design APIs that work as smoothly as possible and don't extend already-long delays--but long latency is, to some degree, unavoidable. – Jonathan Eunice Aug 18 '14 at 23:14

To me the best solution seems to be a method that takes an optional parameter that allows the caller to determine whether or not to include the follower data. In this way the caller can determine if it is appropriate to include the follower data in the call.

In a worst case scenario you have a few extra lines of code that never matter because the call always wants all of the data. However if the caller does not need all of that data and the application sees some performance hits from the collection of the extra data then some fine tuning can be done to allow for the conditional loading of the follower data.

On the other hand if you hard code it to always get the follower data then if it does become a performance issue the change will be more work if you have to make the change to allow for it at the call, or to make a separate call that only gets the user information. Implementing the optional parameter now should be a trivial task, and allow for the best fine tuning of the application with out having to modify your API.

• In the worst scenario it would ask an optional param for every optional data (even if it could be fixed with Enum and EnumSet) i think this solution would be a bit unhandy. I think i will go with the approach of caching the data only when needed. So the api don't add more work to the GC with useless data. – Marco Acierno Aug 18 '14 at 17:12
• @MarcoAcierno - Thats fine if that cached information will never go stale. But if you cache it and the information does go stale then you could be showing followers that are no longer followers, and allowing users that may not be allowed. I do not know your requirements but caching is best done with static data rather than data that can be changed and those changes would have an effect on the operation of the program. – SoylentGray Aug 18 '14 at 17:16
• Yes, but for this problem i already provided a way to the user to force the redownload of the informations where it needs always the most recent version. – Marco Acierno Aug 18 '14 at 17:18
• @MarcoAcierno - So then you are not really gaining much by caching. – SoylentGray Aug 18 '14 at 17:22
• Why? It could re-execute the request and start again using the cache (the cache will be updated with the new data) -- Or show the cached data while the new data is in download – Marco Acierno Aug 18 '14 at 17:23