4

I know that profiling is useful to identify bottlenecks and determining what parts of the code require how much time to execute. The latter isn't always very easy to track in the midst of other paths being executed, so once I decide what I want to optimize it might be problematic to see the improvement in numbers. This is especially true in desktop apps which run constantly and it is difficult to: execute the same path and execute it the same number of times to have reliable comparison.

It won't help me if before optimization the function ran X times and took 500 milliseconds, and after optimization it run Y times and took 400 milliseconds.

In such cases, can I somehow use a profiler to determine improvement or do I have to resolve to other options?

6
  • 3
    So you trust the profiler to identify the bottleneck but not to see if the bottleneck is still there? Commented Feb 18, 2013 at 18:14
  • @AnthonyPegram I guess I've phrased myself incorrectly. I've modified the question to make it clearer. I am not talking about big bottlenecks but rather about not-easily replicable scenarios where the gain might not be as large as in your average bottleneck.
    – Maurycy
    Commented Feb 18, 2013 at 18:39
  • 2
    @MaurycyZarzycki If you can't reliably see the problem (or the effect of the problem) then how do you know you've identified a bottleneck at all?
    – Caleb
    Commented Feb 18, 2013 at 18:43
  • 1
    You can, however, calculate the per-execution time for both versions by dividing time by the number of executions. That's really the metric you want to compare.
    – Blrfl
    Commented Feb 18, 2013 at 18:48
  • Download the 30-day trial of Intel's (buggy) vTune and profile your app with that. It should show you in far more detail than you need what's going on
    – James
    Commented Feb 18, 2013 at 19:52

2 Answers 2

6

once I decide what I want to optimize it might be problematic to see the improvement in numbers.

One has to wonder whether you've really found a bottleneck, and also whether you've really eliminated one, if you can't see the effect in the profiler.

This is especially true in desktop apps which run constantly and it is difficult to: execute the same path and execute it the same number of times to have reliable comparison.

This is one of the things that unit tests can help you with. You should be able to recreate any situation that interests you in a test, even (especially!) those which might be very difficult to replicate in the wild. You can also repeat that test as many times as you want to. If you run the profiler against your test, then, you should be able to see the effect of whatever improvements you make. If you can't measure the effect of the improvement, you really have no grounds for claiming that you've improved the code at all.

2
  • 3
    +1 "If you can't measure the effect of the improvement, you really have no grounds for claiming that you've improved the code at all." Commented Feb 18, 2013 at 19:44
  • You are right, pretty much. I am too used to measuring performance relatively to a different implementation, not as how it relates to the whole application performance. I guess it comes from my game development background where performance is important everywhere. And my microoptimization deviation.
    – Maurycy
    Commented Feb 18, 2013 at 20:12
3

In such cases, can I somehow use a profiler to determine improvement [...]?

Of course. The timings you get not only help to determine improvement but you can see how hotspots move around in the process, getting very informed feedback of how your changes are affecting critical execution paths as well as things like cache misses.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.