Software Engineering Stack Exchange is a question and answer site for professionals, academics, and students working within the systems development life cycle. It only takes a minute to sign up.
Sign up to join this community
I understand that in HPC hybrid systems, for instance a MIC architecture, main memory access is much slower than access to data in own cache or in the cache of another core.
I read that HPC MIC architecture works best with programs with streamed memory access, or negligeable memory access.
When do we say that w program is latency-bound or memory-bound? What is the difference? Are there some HPC architecture which are optimal for this kind of programs?
An application is called memory-bound if it requires a lot of data from memory, so most of its execution time is spent reading and writing data.
Latency-bound can be thought as a subset of the memory-bound category and it occurs primarily when you don't retrieve too much data from memory at once, but you have to wait a lot to get the data close to the processor, in the upper levels of the memory hierarchy.
What developers usually do is to hide this latency through techniques such as software pipelining and data prefetching and to organize the instructions and operations of the applications in a way that the processor does not sit idle waiting for data.
I don't know if there's a specific architecture to tackle memory-bound applications but I know of techniques like the ones mentioned before and that the computer industry is trying to bridge the gap between memory and processor performance by bringing the data closer to the computation (SoC) or developing co-processors that can perform more autonomously from the host (and its memory).
There's a similar thread in this StackOverflow thread. Also, you can take a look a several papers like this one.
As an example, floating-point arithmetic often has high throughput, but also high latency. For example, you might be able to start two multiplications every cycle, but it might take five cycles until the result of a multiplication is available. The first is called throughput (two per cycle), the second is latency.
If you perform calculations so that one operation is dependent on the previous one, you may become latency bound. Say you calculate (x0 + y0) * (x1 + y1) * (x2 + y2) * ... If you do this in a naive way, then each multiplication can only start 5 cycles after the previous one, so you end up with 0.2 multiplications and 0.2 additions per cycle even though the processor could do much more work per cycle. That's code that is latency bound.
Hyperthreading is very useful for latency bound code, because it's very easy for the processor to handle two latency bound threads at the same speed as one. If your code is limited by throughput, hyper threading doesn't help one bit. Basically, the worse the code, the more help do you get from hyper threading.
