Optimizing programs by identifying and taking evaluation (calculation) shortcuts

Question

Recently I am working on a project that involves a lot of simple numerical calculations being applied to large arrays. The numerical values are very simple but there are many different types, such as positive integers, negative integers, small floating point numbers, and so on. SIMD programming techniques (such as SSE) are widely used.

I noticed that a lot of times processing speed can be improved if I can find shortcuts in the calculations. For example, if the inputs are integers and the outputs are also integers, I should try my best to avoid converting to floating point numbers if possible.

For simple chained calculations this is easy. However, when we increase the number of operations that can be combined, we find that we have to manually code those SIMD instructions for each case.

I have studied the approaches taken by various vectorization libraries, and observed that while their approaches are very elegant in code using metaprogramming techniques, most compilers aren't able to generate optimized machine instructions from the results.

If I represent the calculations with command objects, will I have a better chance of matching small sections of the command chain and substituting with shortcuts?

Answer 1

On the assumption that the compiler can optimize out the overhead associated with using command objects by inlining the function calls then this is a perfectly valid optimization technique. It can be retrofitted, but is probably better to plan up front. If you are using C++ (possibly with boost to do some compile time magic for you) and your command objects are small then generally this assumption should hold. I've used this technique myself less in a fairly large computational project that wrote a number of years ago and found that it worked well. One thing to be aware of is that programming in this way can (will) increase the overall complexity of the project and make it harder to read and maintain.