2

I am working with an API but I am unsure what the best way to store the API return values.

Specifically, I am working with the RIOT API just as a small hobby project. I would like to store this information somewhere so that I can not only compare users with each other but also present user statistics without having to call the API each time.

It just doesn't seem right for me to store all this information when it is already there but again, I don't really see the need to constantly call the API for each game that I want to retrieve information from each time the user wants information.

Improve this question
3
  • 2
    What are you doing with the information? Is this an application that can wake up, make all the RIOT API calls, and then reuse that information until the user exits or hits a refresh button? Or does it need to hang around in the background for a while and have some logic to refresh itself periodically? Cache incoherence can be a very hard problem, and you're right to be concerned about it, but (unsurprisingly) it has no one general solution. We'd need far more detail about your use case in order to suggest anything more helpful than "call the API as often as you can until it's too slow".
    – Ixrec
    Commented Nov 9, 2015 at 7:54
  • Often when you find that retrieving information frequently in your program is slowing things up, you add a weak reference cache, meaning if the program has already retrieved it and has the memory to hold it, you'll find it in the cache, otherwise you retrieve it and throw it in the cache. However if concurrency is top priority, there is no other way of doing it. You must retrieve each and every time. So, you should ask yourself what importance concurrency is in your particular case.
    – Neil
    Commented Nov 9, 2015 at 8:36
  • @lxrec The application is a desktop application and needs to hang around in the background. When the user finishes a game, the client will retrieve the game information after a set time.I was thinking that in my case, I would store the frequently retrieved static data locally on the machine for easy reference and have call only the data that is updated only when I need it.
    – Brandyn
    Commented Nov 9, 2015 at 8:40

2 Answers 2

Reset to default
2

Remember the principles of caching. You cache

  1. Data that is close spatially
  2. Data that is close in time

The first is called spatial locality and the second is called temporal locality. A simple example of the first type of data is: If a user asks for a variable part of an object, is it good to fetch the entire object because it is likely that the user also is going to ask for data that is "close" (e.g. in the same object).

A simple example of the second type of caching is that data that is used often, is good to cache. If the user viewed variable A, then he is likely to view variable A again.

So cache your data using memcache, redis, a local hash table or similar, and use the principles of caching to choose which data you use.

You are also fortunate that you can actually measure the hit-rate of you caching once you have it in place. The hit-rate is simply number of hits divided by number of calls.

If you want to study the details of cache memories, a good book is Computer Organization & Design by Patterson /Hennesy.

Improve this answer
1

What is it going to cost you for stale data?, and what does an API call cost?

If you are validating a cellphone number, and you are being charged $10 per click on an enterprise agreement, then probably not more than quarterly.

Credit score ? Probably daily.

Market depth ? your call, trading systems will not cache at all, but a unit trust valuation may only need an hourly update.

With any game data, you do not want to draw attention to yourself as impacting performance, sparingly is a good strategy for anything that is free. Like scraping web pages.

And yes, as intimated in another answer, you can measure the success of your strategy to decrease the age of your data during periods of increase volatility.

A cost/benefit analysis should always be pulled out of your hat to defend whatever decision you make, and if it really matters, give a suitably empowered user the ability to refresh data if absolutely necessary.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.