When deciding what to optimize, always remember Amdahl's law. See the link for precise math; the pithy statement to remember is:

If one part of your program accounts for 10% of its runtime, and you optimize that part to run twice as fast, the program as a whole will only speed up by 5%.

This is why people always say it's not worth optimizing parts of your program that don't occupy more than a few percent of total runtime. But that's just a special case of a more general principle. Amdahl's law tells you that if you need to make the entire program run twice as fast, you need to speed up every piece by an average of 50%. It tells you that if you need to process twenty gigabytes of data, and you don't have twenty gigabytes of RAM, there are only three ways to make that go faster than the time it takes to read twenty gigabytes off the disk: get more RAM, get a faster disk, or make the data smaller.

So what does Amdahl's law say about micro-optimizations? It says that they're maybe worth it if they apply across the board. If you can shave one percent off the runtime of every function in your program, congratulations! You've sped up the program by one percent. Was that worth doing? Well, as a compiler guy I would be delighted to find an optimization that did that, but if you're doing it by hand, I'd say look for something bigger.

When deciding what to optimize, always remember Amdahl's law. See the link for precise math; the pithy statement to remember is:

If one part of your program accounts for 10% of its runtime, and you optimize that part to run twice as fast, the program as a whole will only speed up by 5%.

This is why people always say it's not worth optimizing parts of your program that don't occupy more than a few percent of total runtime. But that's just a special case of a more general principle. Amdahl's law tells you that if you need to make the entire program run twice as fast, you need to speed up every piece by an average of 50%. It tells you that if you need to process twenty gigabytes of data, there are only two ways to make that go faster than the time it takes to read twenty gigabytes off the disk: get a faster disk, or make the data smaller.

So what does Amdahl's law say about micro-optimizations? It says that they're maybe worth it if they apply across the board. If you can shave one percent off the runtime of every function in your program, congratulations! You've sped up the program by one percent. Was that worth doing? Well, as a compiler guy I would be delighted to find an optimization that did that, but if you're doing it by hand, I'd say look for something bigger.