I was reading up about branch predictions and cache misses, and have decided to read
what every programmer should know about memory. But I'm not sure how much low level knowledge do I need to have to write performant code.
One classic example is accessing the memory in the wrong way, accessing column by column when it's row-major would cause a cache miss on every lookup.
I ran a benchmark of 10 000 by 10 000 matrix, and the correct order runs in 97ms while the wrong order runs in 1535ms.
While it's evident that accessing the memory in the right order is much faster, but by how much? In this example, it's only faster by 1.44s. So a program written by a programmer without such memory knowledge is probably only penalized by up to a few seconds.
I have 2 questions.
- How much low level knowledge does a competent programmer need to have?
- How much time saved through optimization is considered worthwhile? At least a minute shaved? Half a minute? Or any amount of time saved?
Some people think that it's totally not worth it to optimize just to save a few seconds or even milliseconds. But what do you all think?