Hot answers tagged

117

In the case of a CPU cache, it is faster because it's on the same die as the processor. In other words, the requested data doesn't have to be bussed over to the processor; it's already there. In the case of the cache on a hard drive, it's faster because it's in solid state memory, and not still on the rotating platters. In the case of the cache on a web ...


75

You implemented a cache (I assume) because the system wasn't performing well enough. That is something that is relevant for the user. Here are things that QA can check: That the system, after the cache was introduced, is still correct. This means they should attempt to trick the cache into providing stale data, which is something QA can verify, and make ...


37

One question is whether the cache itself is really a requirement that should be tested by QA. Caching improves performance, so they could test the difference in performance to ensure it meets some requirement. But good idea to have some testing around caching, whoever is responsible for it. We used performance counters. If your cache system takes advantage ...


33

It is faster because both it is closer and because it is SRAM not DRAM. SRAM is and can be considerably faster than DRAM the values are kept statically (the S in SRAM) so they don't have to be refreshed which takes away cycles. DRAM is dynamic, like tiny rechargeable batteries, you have to regularly recharge the ones so they don't drain away and become ...


32

This is probably too broad for a definitive answer. Personally, I feel that a data access layer is the better place for caching, simply because it is supposed to be very simple - records go in and out and that's it. A business layer implements many additional rules of higher complexity, so it's better if it doesn't also have to manage per-object ...


26

Data access and persistence/storage layers are irresistibly natural places for caching. They're doing the I/Os, making them handy, easy place to insert caching. I daresay that almost every DAL or persistence layer will, as it matures, be given a caching function--if it isn't designed that way from the very start. The problem is intent. DAL and persistence ...


23

Then what? Since nobody said it: Toolbox (broken link). That is if you really really need global variables. Singleton abuse can be avoided by looking at the problem from a different angle. Suppose an application needs only one instance of a class and the application configures that class at startup: Why should the class itself be responsible for being a ...


23

If this class really was as trivial as it appears to be, then there would be no need to worry about violating the SRP. So what if a 3-line function has 2 lines doing one thing, and another 1 line doing another thing? Yes, this trivial function violates the SRP, and so what? Who cares? The violation of the SRP starts becoming a problem when things get ...


22

One thing that should be mentioned explicitly is the impact of the speed of light. In this video Grace Hopper shows a piece of wire about a foot long, which is how far an electrical signal can travel in one nanosecond*. If a CPU is operating at 3GHz, then that implies a distance of 4" per clock cycle. This is a hard physical limit on memory access speeds. ...


18

First if you are concerned about recent (last 5-10 years, since Nahalem?) Intel x86 architecture, then you're a little off in your description of the caches. Each core has their own 128K L1 cache split (64K data / 64K code). Above that, each core has its own L2 cache which basically acts as a buffer between the L1 and L3 cache. Each socket has its own L3 ...


18

Caching on the DAL is straightforward and simple Your DAL is the central data access layer, which means that any and all data access can be controlled through the classes there. As both reading and persisting happens on those layers it is equally easy to clear or update cache entries as changes happen. Caching in the business is flexible Caching on the ...


15

I've been trying to address a similar issue. My users need to be authenticated for every request they make. I've been focusing on getting the users authenticated at least once by the backend app (validation of the JWT token), but after that, I decided I shouldn't need the backend anymore. I chose to avoid requiring any Nginx plugin that is not included by ...


15

Just like using a single database for multiple services, the approach you describe causes a strong coupling between the services. For example, you can not change the data model of one service without having to accordingly change the model in others. This means that both development and deployment are coupled and your microservices in reality become a ...


14

You need a cache busting solution. The role of cache busting is: Rename resources to a unique name depending on their content. Update all references to those resources. In a Grunt-based project it's common to use grunt-rev to ensure that all files that need to be refreshed are given unique names, based on their content. If you ensure that your JSON files ...


11

Having more variables than registers isn't necessarily a problem. If a variable's value isn't used after a certain point in the function, the compiler can use that register for another variable. Even when there's more variables in use at a certain point than there are registers, the compiler will probably do a better job of figuring out the order in which ...


10

A cache line is the smallest unit that you can touch physical memory with. Meaning when you read/write 1 byte, a full cache line containing it is read into the cpu cache and written back. Note that even instructions that bypass the cache to write (ephemeral streaming instructions) write in cache line sizes. Depending on the CPU, cache line sizes are ...


10

As always, one must differentiate: what information is to be stored in the cache? I always go with these simple rules: Information every webserver instance can calculate for itself should go into an in-RAM cache as these need to be accessible ASAP on request but have no need to be shared (never change or do not contain information relevant to other servers)...


8

My suggestion is that instead of worrying about whether to use an external or internal cache, your first concern should be that your booking-service does not care whether or not your are using an external service. That is to say, your booking-service should be caching against an interface with the concrete implementation injected in; it would not know or ...


8

There would have to be quite a discrepancy in the age of the respective components, for a network adapter (and connected infrastructure) to exceed the performance of system RAM, so the premises of this scenario are somewhat contrived. Nevertheless, under TCP the receiver requests data in chunks, the completion of which must be acknowledged before the sender ...


7

PSR-6 recommendation regarding Caching Interface expresses a view I personally prefer: While caching is often an important part of application performance, it should never be a critical part of application functionality. So, I'm in favor of considering it as an optional dependency.


7

You are correctly pointing out a race condition. Cache-aside, as described here and here, is an imperfect abstraction that is not appropriate for all data storage use cases. Consistency. Implementing the Cache-Aside pattern does not guarantee consistency between the data store and the cache. An item in the data store may be changed at any time by an ...


6

You can use if-modified-since + last-modified or if-none-match + etag headers along with the proper cache-control header. (There can be browser bugs, but nothing you cannot manage in recent browsers.) If the files are static, then I suggest you to use if-modified-since, because it can be done automatically with a well configured HTTP server. It should send ...


6

I believe your class is doing one thing; it's a data cache with a timeout. LoadFluffies seems like a useless abstraction unless you call it from multiple places. I think it would be better to take the two lines from LoadFluffies and put them in the NeedsReload conditional in GetFluffies. This would make the implementation of GetFluffies a lot more obvious ...


6

Your class itself seems fine to me, but you're right in that LoadFluffies() does not exactly what the name advertises. One simple solution would be to change the name and move the explicit reloading out of GetFluffies, into a function with an appropriate description. Something like public Fluffies GetFluffies() { MakeSureTheFluffyCacheIsUpToDate(); ...


6

How do I store objects, whose sizes are known only at runtime, contiguously in the memory? The sizes of the objects in one container are identical. Allocate large chunks of memory. Use placement new to initialize objects. char* mem = new char[OBJECT_COUNT*sizeof(Object)]; // ... // Keep track of where the next object can be constructed. Object* obj = new ...


6

I think you're describing two different situations: what do you call data that is not yet fresh, and has never been fresh? Garbage. When you read a value before initialization happens, you read uninitialized data. It's been left in whatever state from whatever touched it last. You can't even call this random because it can't be trusted to be random. It's ...


6

To understand how alignment affects things, let's look at a larger context. First, as you note, 2600 bytes of UTF-8 (or any kind of data) will indeed take 2600 bytes. If you allocate 2600 bytes from the heap using malloc(2600) e.g. in C, then since malloc does not accept alignment information, it will not know that you're intent is to store only individual ...


6

I know that they store messages in cache... but wouldn't just a common database be enough? So at time of writing, Whatsapp has... north of 1.5 billion users (https://www.statista.com/topics/2018/whatsapp/ says 1.5b as of Dec 2017). Multiply that times the number of messages a given user sends and you have a lot of messages. And sure, you can shard the ...


5

There are a lot of good points raised in the other answers, but one factor appears to be missing: address decoding latency. The following is a vast oversimplification of how memory address decoding works, but it gives a good idea of why large DRAM chips ar generally quite slow. When the processor needs to access memory, it sends a command to the memory ...


5

Other answers already covered all the relevant bits: locality (and the associated data transfer cost, bus width and clock, and so on); speed of light (again, associated to transfer costs and bus width and throughput); different memory technology (SRAM vs.DRAM). All of this seen in the light of cost/performance balance. One bit that was left out and it's ...


Only top voted, non community-wiki answers of a minimum length are eligible