Random Cache Expiry

Question

I've been experimenting with random cache expiry times to avoid situations where an individual request forces multiple things to update at once. For example, a web page might include five different components. If each is set to time out in 30 minutes, the user will have a long wait time every 30 minutes. So instead, you set them all to a random time between 15 and 45 minutes to make it likely at most only one component will reload for any given page load.

I'm trying to find any research or guidelines on this topic, e.g. optimal variance parameters. I do recall seeing one article about how Google (?) uses this technique, but can't locate it, and there doesn't seem to be much written about the topic.

Answer 1

Some documents:

• Optimizing Caching from the Let's Make the Web Faster Google series (I assume you meant this one?)
• The Caching Tutorial on mnot.net
• There are some relevant SO questions as well:
  • Set HTTP Caching Expiration, Recommended by Google PageSpeed
  • Set HTTP Caching Expiration, Recommended by Google and Yahoo
• And some other StackExchange-hosted questions as well:
  • What are the Hard and Fast Rules for Cache Control?

Answer 2

Coming back to answer my own question, the main problem here is how to avoid everything always expiring at the same time. If that's allowed to happen, the system will slow down and become congested while it re-populates the cache.

Most of the time, this really isn't a problem in practice. Over time, all the components tend to drift in terms of the time they're expiring. If there are multiple components all being re-built at the same time, it's a code smell because they probably should be cached together as a single component (e.g. if you had a page's unique header, body, and footer cached separately, maybe you can just cache the page itself).

There certainly are times when many things need to be cached at one time, e.g. after starting a system, if we've cleared the whole cache or rotated the cache keys. In this case, it's not usually so bad because the components rapidly fill up, and expiries will subsequently drift apart.

To the extent this is a problem, there's a couple of solutions:

• Simply pick a random cache expiry duration within a range instead of a fixed duration, e.g. a random integer between 15 and 90 minutes instead of 60 minutes.
• Allow stale responses. Just because a cache item has expired, doesn't mean you can't use it if it's still there. Depending on the business needs, it might be acceptable to use it if there's a performance issue fetching the original version after expiry. In HTTP, this is the purpose of "must-revalidate" (if true, it means not to use the cached version after expiry).