I'm referring this chart of latency numbers, attributed to Jeff Dean at Google.
The thing I don't understand is, do these numbers not vary from one set of hardware to the next? How can these be accurate for all different types of RAM, CPU, motherboard, hard drive, etc?
These numbers (also listed on Norvig's Teach yourself Programming in 10 years) are approximate, only useful as (order of) magnitude.
Actually, today's hardware (at least for desktop or laptops) does not vary that much even between a cheap 300€ laptop and a high-end 10k€ workstation. The speed varies by a factor of roughly 2 or 4 at most. Such a workstation can have a larger disk, more cores, cache, and RAM. However, this doesn't have much impact on the raw single-threaded performance.
Look at some figures on http://openbenchmarking.org/ or some CPU comparators.
The so called Moore's law is dying. My 3+ years old desktop at home (an i3770K) could be replaced (today, in march 2016) by some i6700 which is only 20% faster.
The numbers are not meant to be accurate. It is the ratios between the orders of magnitude between tiers that matters.
However, when a disruptive technology appears (e.g. cloud computing, 10GB/100GB ethernet, new networking kernel module, SSD storage networks, virtualization and containerization), these numbers can be invalidated due to new tiers appearing, disappearing, or being shuffled around.
When programming at a very high level - where all of the computation, networking, parsing, etc., are performed using libraries not written by yourself, knowing the performance figures of low-level operations may not help much, since your opportunity to improve each library's performance is rather limited or outright impossible.
Instead, read the performance-related documentation of each library carefully. If a library does not come with those, ask them - make it an issue. Or learn how to benchmark software in the correct way.
Having a basic understanding of latency numbers is important when you are hired by a company that designs and manufactures software components. Compare that to a company that designs and manufactures cars and every component contained within - the proverbial "reinventing the wheel" (rubber, tire pressure, treads, etc.)
Most software companies do not work at the component level - entire functional software systems can be constructed from putting components together. These software companies do not need to focus on how to engineer components in terms of latencies; instead they need to evaluate the quality of the components they choose.
To summarize, (1) it is very possible that you don't need to know the latency numbers; (2) unless you want to be hired by a company that makes software components (libraries), whether for sale or for internal use (as in some of the largest software companies in the world), (3) if you need those numbers, it is your job to do the benchmarks yourself, in a scientifically correct way, or else you shouldn't be working on software components.
Nobody made any claim that these numbers are accurate for any hardware.
However, they are much, much more accurate than blind guesses. Which is what many people unfortunately base their code on.
They aren't perfectly accurate, and they're not really intended to be.
They are (especially on the smaller numbers) a little better than just order of magnitude though. Another point is that it can help make sense of which things are close together, that people sometimes misinterpret as being much further apart than they really are. For one obvious example, quite a few people assume that branch misprediction is frequently a big thing. It can be a big deal if it's repeated a lot, but it's not necessarily worth sacrificing a huge amount anywhere and everywhere else just to get better branch prediction (e.g., if you read from main memory, or even L2 cache to improve branch prediction, it's probably a net loss).
At the same time, yes, orders of magnitude may be the most useful parts. For example, it takes around 100 times longer to access data from main memory than from a register. Yes, on one machine it might be around 97 times longer, and on another it might be closer to 127 times longer. It's almost certainly going to be be closer to 100 than to either 10 or 1000 though.
Personally, I'd tend to think of most of these as being similar to islands in, say, the Pacific Ocean. Hard drive speeds (for example) might be the Hawaiian islands. SSD speeds are the Philippine islands. This is showing the map at a small enough scale to make each of those look like a single point. If we zoom in, that's clearly not true--but the distance between the two chains is many times larger than the distances between the islands in either chain.
Of course the numbers cannot be accurate for every machine. And I guess they never were supposed to. They do show differences in the order of magnitude between several kind of operations, though.
You may find some more useful links and data in the comments of your linked data.
