Do I get valid profiling results with optimization turned off?

Question

I use perf to profile my application. To make interpretation of the resulting call graph easier, I partitioned my core loops using label functions like maptable_appendRange that can be easily searched for in the call graph. Unfortunately, these functions are mostly inlined by the optimizer and therefore do not show up in the call graph.

To fix this, I can compile without optimization (-O0) and everything is fine and profiling with my label functions is a bliss. On the other hand, optimization has quite an impact on my code's overall performance and I'm not sure if profiling without optimization will be representative of actual performance.

So here is my question: can I expect representative results when profiling unoptimized code?

Answer 1

You gotta ask why you are profiling.

1. To get general performance measurements.

2. To find ways to make the code go faster yet.

If the answer is 1, then by all means, run your profiler on the optimized code.

If the answer is 2, then don't. Here's why. There are two kinds of speedups, the ones you can do, and the ones the compiler can do.

The compiler cannot find or fix the speedups you can do, such as minimizing memory allocation, memoizing functions that get called with repeat arguments, avoiding innocent-looking library calls that end up doing breezy I/O, on and on.

There is no speed bug you can fix that is made any more evident by the optimizer.

What it can do is make them harder to find by scrambling the code, randomly inlining, getting rid of stack frames, etc.
And keep in mind, the only improvement it can actually make is at the bottom of the call stack, and only if it's in your code. Anything it does higher up the stack is of minimal/negligible benefit.

The strategy some people use is this, and the statistical reason why it works is here. After having done that enough to make the code blazingly fast without the optimizer, then turn on the optimizer to make it faster yet.

Answer 2

If you want the profiling results to be meaningful, you should use the optimization flags that you intend to use in production.

That said, you can tell most compilers to never inline specific functions. For example, if you use gcc you could mark your label functions with __attribute__((noinline)):

__attribute__((noinline)) int func()
{
    ...
}

(Other compilers usually provide similar extensions.) This should allow the label functions to show up in the profile even when optimizations are enabled. Keep in mind that the attribute is gcc-specific, so remove it once you're done profiling, or wrap it in a macro that evaluates to nothing if a different compiler is used.

Of course, if the inlining of these specific functions actually has a significant impact on performance, the profiling results might still not be fully representative.