14

I've been experimenting with random cache expiry times to avoid situations where an individual request forces multiple things to update at once. For example, a web page might include five different components. If each is set to time out in 30 minutes, the user will have a long wait time every 30 minutes. So instead, you set them all to a random time between 15 and 45 minutes to make it likely at most only one component will reload for any given page load.

I'm trying to find any research or guidelines on this topic, e.g. optimal variance parameters. I do recall seeing one article about how Google (?) uses this technique, but can't locate it, and there doesn't seem to be much written about the topic.

2
  • Can you reword this as a question? It's not clear what you're expecting for an answer.
    – Derek
    Jun 7, 2012 at 16:21
  • Okay, that's done.
    – mahemoff
    Jun 8, 2012 at 8:07

2 Answers 2

5

Some documents:

3
  • 3
    Thanks for these links, but none of them mention randomness.
    – mahemoff
    Jun 8, 2012 at 8:04
  • 2
    I know how old this question is, but Matthew Neale from EstarOnline Limited did just this (PDF): estaronline.com/images/assetimages/… Apr 3, 2018 at 4:53
  • @NickBedford This is the answer I was looking for. Your comment should be the accepted answer. Mar 20, 2020 at 22:19
1

Coming back to answer my own question, the main problem here is how to avoid everything always expiring at the same time. If that's allowed to happen, the system will slow down and become congested while it re-populates the cache.

Most of the time, this really isn't a problem in practice. Over time, all the components tend to drift in terms of the time they're expiring. If there are multiple components all being re-built at the same time, it's a code smell because they probably should be cached together as a single component (e.g. if you had a page's unique header, body, and footer cached separately, maybe you can just cache the page itself).

There certainly are times when many things need to be cached at one time, e.g. after starting a system, if we've cleared the whole cache or rotated the cache keys. In this case, it's not usually so bad because the components rapidly fill up, and expiries will subsequently drift apart.

To the extent this is a problem, there's a couple of solutions:

  • Simply pick a random cache expiry duration within a range instead of a fixed duration, e.g. a random integer between 15 and 90 minutes instead of 60 minutes.
  • Allow stale responses. Just because a cache item has expired, doesn't mean you can't use it if it's still there. Depending on the business needs, it might be acceptable to use it if there's a performance issue fetching the original version after expiry. In HTTP, this is the purpose of "must-revalidate" (if true, it means not to use the cached version after expiry).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.