8

I use perf to profile my application. To make interpretation of the resulting call graph easier, I partitioned my core loops using label functions like maptable_appendRange that can be easily searched for in the call graph. Unfortunately, these functions are mostly inlined by the optimizer and therefore do not show up in the call graph.

To fix this, I can compile without optimization (-O0) and everything is fine and profiling with my label functions is a bliss. On the other hand, optimization has quite an impact on my code's overall performance and I'm not sure if profiling without optimization will be representative of actual performance.

So here is my question: can I expect representative results when profiling unoptimized code?

Improve this question
7
  • profiling is profiling, it is against the code you built no matter how you built it. If optimized then that is what you are measuring, if not optimized, then that is what you are measuring. If you are wanting to use one measurement of one thing to mean something for the other, that makes no sense. You nave to measure the thing you are using and use those results, cant measure something else. The price of tea in china has nothing to do with the temperature in Texas.
    – old_timer
    Aug 28, 2016 at 14:51
  • A better analogy is having the same driver (source code) do laps in a minivan and laps in sports car and somehow try to use the minivan lap time to talk about the sports car lap times.
    – old_timer
    Aug 28, 2016 at 14:53
  • To stick with your analogy, in my use case I am interesed in the driver's performance and not so much in the cars.
    – Johannes Luong
    Aug 28, 2016 at 15:02
  • which you really cant do in this situation, the output is expected to be vastly different in ways that are somewhat predictable with experience but that doesnt mean the profile will see that. We can tell what the optimizer will do with local variables and if it will inline functions or not, where the cliffs are with respect to all in registers or falling off the cliff and having to use the stack, etc. But the profile is of both micro and macro taken at the same time ideally. A sports car vs a yugo.
    – old_timer
    Aug 28, 2016 at 15:07
  • No matter how great the driver they cannot make any car perform through the same corner on the same track in a proportional way. In one vehicle the straights are the problem in another it may be the corners. Sometimes the profiler will find the same bottleneck but that may be just dumb luck for that one application, for another it may find another place.
    – old_timer
    Aug 28, 2016 at 15:09

2 Answers 2

Reset to default
8

You gotta ask why you are profiling.

  1. To get general performance measurements.

  2. To find ways to make the code go faster yet.

If the answer is 1, then by all means, run your profiler on the optimized code.

If the answer is 2, then don't. Here's why. There are two kinds of speedups, the ones you can do, and the ones the compiler can do.

The compiler cannot find or fix the speedups you can do, such as minimizing memory allocation, memoizing functions that get called with repeat arguments, avoiding innocent-looking library calls that end up doing breezy I/O, on and on.

There is no speed bug you can fix that is made any more evident by the optimizer.

What it can do is make them harder to find by scrambling the code, randomly inlining, getting rid of stack frames, etc.
And keep in mind, the only improvement it can actually make is at the bottom of the call stack, and only if it's in your code. Anything it does higher up the stack is of minimal/negligible benefit.

The strategy some people use is this, and the statistical reason why it works is here. After having done that enough to make the code blazingly fast without the optimizer, then turn on the optimizer to make it faster yet.

Improve this answer
1
  • 1
    I think your answer comes close to what I thought off. My application uses parallel task execution and I want to find scalability bottlenecks in my algorithms. These bottlenecks are caused by algorithmic choices whose tradeoffs will not be altered drastically by optimization. Therefore I probably can profile unoptimized code.
    – Johannes Luong
    Aug 23, 2016 at 13:45
7

If you want the profiling results to be meaningful, you should use the optimization flags that you intend to use in production.

That said, you can tell most compilers to never inline specific functions. For example, if you use gcc you could mark your label functions with __attribute__((noinline)):

__attribute__((noinline)) int func()
{
    ...
}

(Other compilers usually provide similar extensions.) This should allow the label functions to show up in the profile even when optimizations are enabled. Keep in mind that the attribute is gcc-specific, so remove it once you're done profiling, or wrap it in a macro that evaluates to nothing if a different compiler is used.

Of course, if the inlining of these specific functions actually has a significant impact on performance, the profiling results might still not be fully representative.

Improve this answer
3
  • This is common wisdom, taught in schools or by people with high rep scores on stackoverflow. But it is based on a hidden assumption, namely that there are no further speedup opportunities in the code, and all you want to do is time things. If you are looking for those speedups, the code's being optimized does not help. It can steer you away from some low-level routine that the compiler can optimize, but other than that, it doesn't help in telling you what you can fix.
    – Mike Dunlavey
    Sep 6, 2016 at 1:19
  • @MikeDunlavey: How about just adding -fno-inline or similar on top of normal options to keep the profile accurate?
    – Deduplicator
    Oct 29, 2016 at 16:58
  • @Deduplicator: You can do that if you want, but it still scrambles the code and keeps some variables in registers, even if it's in functions that are nowhere near the bottom of the call stack, so optimizing them doesn't help. If you want line-level information, this scrambling makes it impossible to get. Besides, if by "accurate" you mean "measuring time with precision", that does not tell you what speed bugs you have. For that, you need precision of location/situation, not precision of timing.
    – Mike Dunlavey
    Oct 29, 2016 at 20:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.