Cache invalidation on multiple servers

Question

I have 6 web servers which are giving me problems due to cache inconsistency. I am thinking of building a cache invalidation service such that there is a topic on which all the servers can publish a message to invalidate an object. I am considering using the Amazon SNS for the making the topic.

Now for the servers to receive the invalidation messages, I am confused between the following:

1. Should I be using the SQS queues for the servers to receive messages.
2. Should I be using HTTP endpoints and then build an api on that route that invalidates the cache.

Could you please highlight the pros and cons of both of these approaches or any other approach that might benefit me.

----------UPDATE---------

If I use NGINX to redirect any request to one of the 6 servers. If I am using the HTTP endpoint, the topic will end up hitting only one of the servers. I also am not sure which port my application would be running on. Could you suggest me a way to by-pass the NGINX server or find out the port on the fly and then hit the servers.

Answer 1

According to the comments, you have a caching system which relies on:

• A common Redis cache service used by all the servers,

• An in-memory cache on every server which somehow duplicates some of the Redis data. Information stored in this cache is subject to inconsistency, and you are looking for a way to prevent this inconsistency.

First and foremost, you will probably only need the Redis cache. If configured correctly, it is fast enough to serve all six servers. If you only guess that it will make things faster by having an additional in-memory caching, then you're doing premature optimization. Instead, get rid of in-memory cache and think about it only if and when a profiler will identify Redis as a bottleneck.

If you already profiled your app and identified Redis as the bottleneck, you have a choice:

• You may be able to change Redis configuration or modify the network or replace the hardware, depending on the actual issue you are encountering.

• You may put Redis on several servers and let existent products deal with subjects such as failover, distributed cache invalidation, etc.

• You may ask others, such as Amazon, to handle caching for you. This gives you a huge flexibility in terms of scale you need at a given moment.

• You may finally use memcached or similar distributed caching solutions which would be installed on each of the six servers. Make sure you do use an existent product, and not reinvent your own, since creating your own could be very tricky; cache invalidation is one of the tricky parts of it, as you have noticed.