0

I was coding some functions in C++ and wondered how different versions of those functions would affect generated assembly code. I put different versions into the Godbolt Compiler Explorer Tool and looked at the generated assembly. It was an interesting experience to see those differences and how some versions that seemed to be more efficient take up much more assembly lines than more verbose ones and in contrast some more low level versions take up more than some "mid high level" versions (contrary to my expectations).

As one cannot judge the performance of those outputs by just looking at the line count I wondered how one can roughly estimate the performance difference between different versions?

How can I analyse different outputs to see more easily if some code output contains more potential expensive ASM calls than another output or do I have to learn assembly first in order to do this?

  • 4
    The instruction set manual for a processor will typically list how many cycles each instruction will take, which can serve as a first approximation of performance in absence of cache misses, branch mispredictions, or pipeline stalls. You can then count how many cycles a piece of code will take. But this is really tricky. Some instructions may have a few cycles of latency until the result becomes available, that means you can execute other instructions for “free” in the meanwhile. – amon Sep 10 '17 at 21:27
  • 3
    Or, you just measure how long it takes to execute, the same way you would any other piece of code. But yeah, if you're doing some serious micro-optimization, the first place I'd look is in the microprocessor's instruction set specifications. – Robert Harvey Sep 10 '17 at 21:28
  • 2
    It is called micro-benchmarking. Execute the said code in a loop for billions (note: millions aren't enough) of times. This requires lots of computer architecture knowledge to do it right, because it's full of pitfalls, and it takes expert preparation to minimize interference and inaccuracies from many sources. Agner Fog has a website that talks about some of these things. At the minimum, the code needs to be tested on each vendor (e.g. Intel, AMD) and each family of CPUs (e.g. Ryzen, Haswell, Skylake, etc.) Benchmark results from one vendor/CPU generation is not valid for another. – rwong Sep 10 '17 at 22:00
  • 4
    // or do I have to learn assembly first in order to do this? // Definitely. As soon as you rearrange the code by e.g. calling it from a micro-benchmarking loop, or using the code in a real-world application, the compiler isn't guaranteed to emit the same piece of assembly code anymore. So if you are interested in performance of machine code, you have to know assembly language as well as compilers. You might not need to know how to write either, but you will need some intuitive understanding of what they do. – rwong Sep 10 '17 at 22:15
  • 4
    @amon: The instruction set manual for the 68020 processor (primitive compared to current designs) says paraphrased: "It is very difficult to calculate the exact execution time of some code, even if you have a deep understanding of the actual implementation of the processor". – gnasher729 Sep 10 '17 at 22:25
7

As others have pointed out, just counting clock cycles of assembler instructions will not give you a decent result for most modern CPU architectures. The execution time of a fixed piece of machine code can vary on different CPU platforms, even if the code is exactly the same. So the only reliable way of comparing the performance of such code snippets is

  • get the compilers you want to compare
  • get one or more hardware & OS platforms for the comparisons
  • embed the code snippets you want to compare into some benchmark program (in its simplest form: just a loop which calls the snippet several million times and measures the total running time)
  • compile the benchmark with each of the compilers, using the available optimization options (maybe tailored for each of the platforms you want to compare)
  • then run the programs and measure on each platform

Obviously, learning assembly is not mandatory for this process (but it will probably be required if you want to understand the root cause for the performance differences you will observe).

That is probably not the answer you wanted to hear, I guess you were expecting a reference to some kind of web service like Mr Godbolt's site which provides you with the tools and hardware, so you don't have to buy and install them by yourself. However, I don't think there is a free service available: sensible performance comparisons need real, expensive hardware, not some cheap virtual machine replacement, and any company which can do this for you will probably try to get a decent return on their investment.

| improve this answer | |
4

There are two flaws in your process:

  • You are testing on a specific tool (Godbolt Compiler Explorer Tool) and looking at its output, which will surely differ from the output of a production-quality compiler (and that also highly depends on the optimization settings you choose).
  • With modern CPU's, it's virtually impossible to predict the run time from just reading the assembly code.

And finally, the general advice on performance optimization: do it only when you found it's necessary.

  • Start with writing well-structured, readable code.
  • Decide whether there's any performance problem. And "if it ain't broken, don't fix it".
  • If there's a problem, use a profiler to find out the real bottleneck (I bet, you'll be surprised where it is).
  • Find a solution for that bottleneck (maybe by micro-optimization, maybe by changing your algorithm...)
  • Profile again, to check if your change really worked.
  • If still too slow, repeat with the next bottleneck.
| improve this answer | |
  • 8
    Godbolt uses the production-quality-compilers to generate its output. It is a compiler comparison website. – Caleth Sep 11 '17 at 8:29
2

First of all: performance depends on the optimization that the compiler would make on your code, so you may want to test on the compiler you will use to build the final program.

Moreover, you will need to test your code for every major producer and chipset because the assembly language is different and similar instructions can be handled quite differently.

| improve this answer | |
2

I was coding some functions in C++ and wondered how different versions of those functions would affect generated assembly code.

And now you know its difficult. Was this just idle curiosity and do you /really/ need to know how fast it is.

I've had people to ask me which is quicker

x = x + 1; x += 1; or x++;

Any decent compiler will probably generate the same code. And your program's got a serious problem if incrementing a variable is a performance issue...

| improve this answer | |
  • No it was just personal interest, nothing to do with micro optimizations. Sometimes it is quite interesting what code compilers generate. – TorbenJ Sep 11 '17 at 16:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.