Where is a good place to start to learn about custom caching in .Net

Question

I'm looking to make some performance enhancements to our site, but I'm not sure exactly where to begin. We have some custom object caching, but I think that we can do better.

Our Business

We aggregate news stories on a news type of web site. We get approximately 500-1000 new stories per week. We have index pages that show various lists of the items and details pages that show the individual stories.

Our Current Use case: Getting an Individual Story

1. User makes a request
2. The Data Access Layer(DAL) checks to see if the item is in cache and if item is fresh (15 minutes).
3. If the item is not in cache or is not fresh, retrieve the item from SQL Server, save to cache and return to user.

Problems with this approach

• The pull nature of caching means that users have to pay the waiting cost every time that the cache is refreshed. Once a story is published, it changes infrequently and I think that we should replace the pull model with something better.

My initial thoughts

• My initial thought is that stories should ALL be stored locally in some type of dictionary. (Cache or is there another, better way?). If the story is not found, then make a trip to the database, update the local dictionary and send the item back.
• Since there may be occasional updates to stories, this should be an entirely process from the user.
• I watched a video by Brent Ozar, How StackOverflow Scales SQL Server, in which Brent states "the fastest database query is the one that you don't make".

Where do I start?

At this point, I don't know exactly what the solution is. Is it caching? Is there a better way of using local storage? Do I use a Dictionary, OrderedDictionary, List ? It seems daunting and I'm just looking for some good starting points to learn more about how to do this type of optimization.

Answer 1

If you were to choose a caching system:

Did you check the following article available at MSDN: http://msdn.microsoft.com/en-us/library/ee817645.aspx ?

It is outdated ("retired" as they say) but still a good read, particularly the second chapter for you since you did not yet made up your mind on the technique/technology you want to use. The following techniques are mentioned:

• Using the ASP.NET Cache
• Using Remoting Singleton Caching
• Using Memory-Mapped Files
• Using Microsoft SQL Server 2000 or MSDE for Caching
• Using Static Variables for Caching
• Using ASP.NET Session State
• Using ASP.NET Client Side Caching and State
• Using Internet Explorer Caching

Also, I've never used it, but I heard a lot of good stuff about Memcached, there is an interesting answer to the Memcached with Windows and .NET question that explains how it might work along a .NET environment.

However, I don't know how long is a story, how many bits it would take, but 520000 in ten years does not seem that much. Maybe Memcached is overkill in that case, I'm not sure...