Hot answers tagged

112

In the case of a CPU cache, it is faster because it's on the same die as the processor. In other words, the requested data doesn't have to be bussed over to the processor; it's already there. In the case of the cache on a hard drive, it's faster because it's in solid state memory, and not still on the rotating platters. In the case of the cache on a web ...


74

You implemented a cache (I assume) because the system wasn't performing well enough. That is something that is relevant for the user. Here are things that QA can check: That the system, after the cache was introduced, is still correct. This means they should attempt to trick the cache into providing stale data, which is something QA can verify, and make ...


37

One question is whether the cache itself is really a requirement that should be tested by QA. Caching improves performance, so they could test the difference in performance to ensure it meets some requirement. But good idea to have some testing around caching, whoever is responsible for it. We used performance counters. If your cache system takes advantage ...


34

In addition to Martijn's comments I'd add: Warm up your JVM. Bytecode starts starts off being interpreted for Hotspot and then gets compiled on the server after 10K observations. Tiered Compilation can be a good stop gap. Classloading is a sequential process that involves IO to disk. Make sure all the classes for your main transaction flows are loaded ...


33

It is faster because both it is closer and because it is SRAM not DRAM. SRAM is and can be considerably faster than DRAM the values are kept statically (the S in SRAM) so they don't have to be refreshed which takes away cycles. DRAM is dynamic, like tiny rechargeable batteries, you have to regularly recharge the ones so they don't drain away and become ...


29

This is probably too broad for a definitive answer. Personally, I feel that a data access layer is the better place for caching, simply because it is supposed to be very simple - records go in and out and that's it. A business layer implements many additional rules of higher complexity, so it's better if it doesn't also have to manage per-object ...


24

Data access and persistence/storage layers are irresistibly natural places for caching. They're doing the I/Os, making them handy, easy place to insert caching. I daresay that almost every DAL or persistence layer will, as it matures, be given a caching function--if it isn't designed that way from the very start. The problem is intent. DAL and persistence ...


23

If this class really was as trivial as it appears to be, then there would be no need to worry about violating the SRP. So what if a 3-line function has 2 lines doing one thing, and another 1 line doing another thing? Yes, this trivial function violates the SRP, and so what? Who cares? The violation of the SRP starts becoming a problem when things get ...


22

There are a bunch of things to be aware of yes. I'm in Crete at the moment with limited net access so this will be (fairly) short. Also, I'm not a low-latency expert, but several of my colleagues play one in real life :-). You need to appreciate Mechanical Sympathy (a term coined by Martin Thompson). In other words you need to understand what your ...


21

One thing that should be mentioned explicitly is the impact of the speed of light. In this video Grace Hopper shows a piece of wire about a foot long, which is how far an electrical signal can travel in one nanosecond*. If a CPU is operating at 3GHz, then that implies a distance of 4" per clock cycle. This is a hard physical limit on memory access speeds. ...


19

Then what? Since nobody said it: Toolbox. That is if you want global variables. Singleton abuse can be avoided by looking at the problem from a different angle. Suppose an application needs only one instance of a class and the application configures that class at startup: Why should the class itself be responsible for being a singleton? It seems quite ...


18

If you want true Unit Tests, then you have to mock the cache: write a mock object that implements the same interface as the cache, but instead of being a cache, it keeps track of the calls it receives, and always returns what the real cache should be returning according to the test case. Of course the cache itself also needs unit testing then, for which you ...


17

Keep your data small if possible Keep things that will be accessed together (or right after another) next to each other in memory Learn about your compiler's optimization parameters Read What every programmer should know about memory for more details than you could ever want


14

I guess the best answer is that it depends. In my experience there are a lot of factors that go into choosing caching algorithms. Factors to consider Read/Write Balance. (What percentage of accesses are reads vs writes) Amount of cache. Type of media behind the cache. (Are they slow SATA drives or fast SSD drives?) Hits vs Misses. (How often are things ...


14

You need a cache busting solution. The role of cache busting is: Rename resources to a unique name depending on their content. Update all references to those resources. In a Grunt-based project it's common to use grunt-rev to ensure that all files that need to be refreshed are given unique names, based on their content. If you ensure that your JSON files ...


14

Caching on the DAL is straightforward and simple Your DAL is the central data access layer, which means that any and all data access can be controlled through the classes there. As both reading and persisting happens on those layers it is equally easy to clear or update cache entries as changes happen. Caching in the business is flexible Caching on the ...


13

First if you are concerned about recent (last 5-10 years, since Nahalem?) Intel x86, then your architect is a little off. Each core has their own 128K L1 cache split (64K data / 64K code). Above that, each core has its own L2 cache which basically acts as a buffer between the L1 and L3 cache. Each socket has its own L3 cache (up to 20MB, I think). The L1 and ...


12

The intricacy of this issue has been beyond human comprehension these days. (It has been that way since the last 5 years.) Combine that with short-vector parallelism (SIMD) and you have a sense of hopeless that optimizing code by hand is no longer economically feasible - not that it's not possible, but it would not be cost-effective anymore. The current ...


12

If you wan to use a LRU eviction cache (Least Recently Used eviction), then probably a good combination of data structures to use is: Circular linked list (as a priority queue) Dictionary This is why: The linked list has a O(1) insertion and removal time List nodes can be reused when the list is full and no extra allocations need to be performed. This ...


12

Is unanswerable, except to say it depends. There are a lot of factors which will determine which approach is going to be the best in your case, e.g.: Is it normal for created objects to be retrieved shortly after they are created? What's the ratio of updates to accesses? Re. deciding you need a cache: If you're optimising without data then yes, it's ...


11

Having more variables than registers isn't necessarily a problem. If a variable's value isn't used after a certain point in the function, the compiler can use that register for another variable. Even when there's more variables in use at a certain point than there are registers, the compiler will probably do a better job of figuring out the order in which ...


10

My recommendation is to look at your usage profile and your requirements for the cache. I can see no reason why you would leave stale data in memcached. I think you have picked the right approach ie: update the DB. In any case, you're going to need a wrapper on your DB update (which you've done). Your code to update the User in the DB and in-RAM should ...


10

The Single Responsibility Principle is your best friend here. First of all, move AllFromCache() into a repository class and call it GetAll(). That it retrieves from the cache is an implementation detail of the repository and shouldn't be known by the calling code. This makes testing your filtering class nice and easy. It no longer cares where you're ...


9

Unless I'm misunderstanding the question, I don't think that this is an appropriate scenario for caching. Cached data normally has at least one of the following attributes (usually all of them): Expensive to retrieve or compute; Highly static - may change occasionally but very rarely; Non-critical - OK if the requester sees stale data. It doesn't sound ...


9

"Are there any reasons not to do this?" Scalability. Sure, when you're using it now, the data is under a few MB. Will that always be the case? I don't think so. Especially if you expect other people to use this system. You're re-inventing the wheel. If the content is stored in files, you should just let Apache (or whatever other web server you're using) ...


9

First you last question: Why Redis/memcached? No, they're not (usually) faster than simple in-process dictionaries. The advantage comes when you have several worker processes, or even many app-layer machines. In that case, instead of each process having its own small cache, they all share a single big (distributed) cache. With bigger caches, you get ...


9

You need caching when you have a problem that can be solved by caching. That problem may be too much processor usage on the DB; if it is then MySQL caching is going to help you a lot -- but maybe not enough, it depends what else is going on. Or it could be that your network connection from the application instances to the DB are getting overloaded. In that ...


9

As always, one must differentiate: what information is to be stored in the cache? I always go with these simple rules: Information every webserver instance can calculate for itself should go into an in-RAM cache as these need to be accessible ASAP on request but have no need to be shared (never change or do not contain information relevant to other servers)...


8

Assuming you know almost nothing about the application you're going to develop, you should know more about it before actually choosing and implementing a cache system. In other words, there are no default implementations: some are good for some purposes, and are totally bad for others. For example, take just two implementations: Least Recently Used and ...


8

I have experimented with different approaches to object caching, and I see advantages to an approach where collections are cached as references rather than actual objects. An example: User GetUser(int ID); ICollection<User> GetRecentUsers(int amount); ICollection<User> GetActiveUsers(int amount); void Update(User user); In this example, if all ...


Only top voted, non community-wiki answers of a minimum length are eligible