7

This has been something that's been bothering me for a while: What makes an application memory bandwidth bound?

For example, take this monstrosity of a computer that calculated the 5 trillionth digit of pi (and later 10 trillionth digit). I was surprised that they choose the lower but faster 98 GB RAM at 1066 MHz instead of the larger but slower 144 GB at 800 MHz. This is especially surprising considering they are using 22 TB HD array to store the results from computation; more RAM means less need for hard drives.

Maybe its because I don't write applications for HPC servers, but how would RAM be the bottleneck? Are there any other non-HPC applications that usually run into this problem?

  • Apparently their algorithm depends more on computational power than it does on memory size. Interesting that you don't need a supercomputer anymore to do this sort of thing. – Robert Harvey Mar 19 '12 at 19:55
  • Are you sure it was the increase of bandwidth and not the decrease of latency which was sough? (The only answer now is targetted at latency and latency is often a biggest problem than bandwidth.). – AProgrammer Mar 19 '12 at 20:31
10

The answer to that is fairly simple: main memory is glacially slow compared to the CPU. To quote from another question in a related area, approximate speeds for a modern CPU are:

L1 CACHE hit, ~4 cycles
L2 CACHE hit, ~10 cycles
L3 CACHE hit, line unshared ~40 cycles
L3 CACHE hit, shared line in another core ~65 cycles
L3 CACHE hit, modified in another core ~75 cycles remote
L3 CACHE ~100-300 cycles

Then, do the math against the DRAM vs a 2.5GHz CPU: for a single cycle the 800MHz you are talking ~ 3-4 CPU cycles. Actual timing of memory access is more complex, but adds up to a lot. The 800MHz RAM there is going to be something like 5-5-5-16, or 124 CPU cycles.

That is still in an ideal world, too: once you add in bus bandwidth, memory contention from multiple accesses to the same chips, cache overheads, and everything else you are talking about access to main memory being thirty to fifty times slower than accessing data in cache.

Disk is, of course, insanely slower. In the analysis of this problem, the question is if the performance cost - RAM that is 75% the speed - is worth the benefit - caching ~ 70% of the data ahead of disk.

To answer the question about where else you run into this: the most common place would be your graphics card, where memory bandwidth is one of the significant constraints in overall fill rate. Improvements there help substantially in increasing overall polygon count in games, etc.

Otherwise, you run into this every day, but only in small ways. You pay that 30-fold cost to write data out to DRAM, and can absolutely see and measure than in all sorts of software that ends up showing a huge performance cliff as the input size increases. Video compression and image processing are common places that this consideration can result in visible performance changes.

  • Hmm, haven't seen that list before. I didn't know math heavy applications (Pi calculator, compression, processing) could thrash the RAM so much that the processor is waiting too much. Thanks for the answer – TheLQ Mar 20 '12 at 14:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.