the term "client" used here is not referring to client's browser, but client server

Before cache workflow

1. client make a HTTP request -->
2. server process -->
3. store parsed results into memcache for next use (cache indefinitely) -->
4. return results to client -->
5. client get the result, store into client's local memcache with TTL

After cache workflow

1. another client make a HTTP request -->
2. memcache found return memcache results to client -->
3. client get the result, store into client's local memcache with TTL

TTL = time to live

Is possible for me to know when the data was updated,
and to expire relevant memcache(s) accordingly.

However, the pitfalls on client site cache TTL

  1. Any data update before the TTL is not pick-up by client memcache.
  2. In reverse manner, where there is no update, client memcache still expire after the TTL
  3. First request (or concurrent requests) after cache TTL will get throttle as it need to repeat the "Before cache workflow"

In the event where client require several HTTP requests on a single web page,
it could be very bad in performance.

Ideal solution should be client to cache indefinitely until further notice.

Here are the three proposals about futher notice

Proposal 1 : Make use on HTTP header (current implementation)

1. client sent HTTP request last modified time header
2. server check if last data modified time=last cache time return status 304
3. client based on header to decide further processing

- save some parsing for client
- lesser data transfer

- fire a HTTP request is still slow
- server end still need to process lots of requests

Proposal 2 : Consistently issue a HTTP request to check all data group last modified time

1. client fire a HTTP request
2. server to return last modified time for all data group
3. client compare local last cache time with the result
4. if data group last cache time < server last modified time 
   then request again for that data group only

- only fetch what is no up-to-date
- less requests for server

- every web page require a HTTP request

Proposal 3 : Tell client when new data is available (Push)

 1. when server end notice there is a change on a data group
 2. notify clients on the changes
 3. help clients to fetch again data
 4. then reset client local memcache after data is parsed

- let the cache act/behave like a true cache

- encourage race condition

My preference is on proposal 3,
and something like Gearman could be ideal
Where there is a change, Gearman server to sent the task to multiple clients (workers).

Am I crazy?
(I know my first question is a bit crazy)

  • 1
    Yeah, my preference is on 3 as well. It's much more RESTful than the other two.
    – CamelBlues
    Commented Sep 9, 2011 at 22:26

2 Answers 2


Proposal 3 is certainly the most efficient, however it is also the most time consuming to implement. Memcache doesn't include this feature, so you'd either have to find some other framework or roll your own (using Gearman in this case). It's not super hard, but it's not trivial and it adds a significant moving part to your system that can fail. I'd make sure to d all the appropriate research before committing to any decision. You might find that proposal 1/2 won't be a huge bottleneck after all and you could save yourself several days of time.


You know your preference, so it is most likely the correct one for you situation. I would suggest you go for it.

HTTP is designed to be stateless. HTTP also has a well defined standard of caching. http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html

Of course, HTTP is also designed to separate client and server implementations. If you can control both, you could create your own lightweight protocol that would allow your preferred method with the push technology.

The only question I could find was "Am I crazy?" And the answer is no, these are all valid suggestions for different scenarios, depending on multiple factors like load, time sensitivity of the data, and many I've haven't though of.

  • 1
    the client is not referring to browser
    – ajreal
    Commented Sep 9, 2011 at 18:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.