Coming back to answer my own question, the main problem here is how to avoid everything always expiring at the same time. If that's allowed to happen, the system will slow down and become congested while it re-populates the cache.
Most of the time, this really isn't a problem in practice. Over time, all the components tend to drift in terms of the time they're expiring. If there are multiple components all being re-built at the same time, it's a code smell because they probably should be cached together as a single component (e.g. if you had a page's unique header, body, and footer cached separately, maybe you can just cache the page itself).
There certainly are times when many things need to be cached at one time, e.g. after starting a system, if we've cleared the whole cache or rotated the cache keys. In this case, it's not usually so bad because the components rapidly fill up, and expiries will subsequently drift apart.
To the extent this is a problem, there's a couple of solutions:
- Simply pick a random cache expiry duration within a range instead of a fixed duration, e.g. a random integer between 15 and 90 minutes instead of 60 minutes.
- Allow stale responses. Just because a cache item has expired, doesn't mean you can't use it if it's still there. Depending on the business needs, it might be acceptable to use it if there's a performance issue fetching the original version after expiry. In HTTP, this is the purpose of "must-revalidate" (if true, it means not to use the cached version after expiry).