I have learned that new Intel processors can do "dynamic execution" (meaning out-of-order execution of instructions). How do programs written in a high level language take advantage of this? Does the compiler modify the code so that they can benefit?

Every year new processor architectures come out but languages like c,c++ remain same. How do programs written in these languages benefit from this new technology?

  • 5
    Out-of-order execution isn't anything new. The CDC 6600 first did it in 1964, which makes Intel a bit of a noob only having started doing it with the P6 in 1995. – Blrfl Mar 4 '17 at 17:23
  • Heard about Meltdown and Spectre? That's how! :) – mlvljr Mar 27 '18 at 4:03

Your program doesn't need to do anything in particular to take advantage of out-of-order execution. The processor will just perform more instructions per cycle because it can start another instruction long before the previous one is finished.

Optimising compilers of course need to know how the processor works to produce optimal code. With out of order execution, sometimes it means that taking extra care is not necessary anymore. Take a sequence of instructions x = a + b + c; y = d + e + f;

Without OoO execution, you would compile this to tmp1 = a+b; tmp2 = d+e; x = tmp1 + c; y = tmp2 + f; because that way the additions can be done in parallel. With out of order execution they are performed in parallel anyway, so the compiler need not bother.

On the other hand, because you can do more operations per cycle, your code may now be limited by latency. Improving one thing that limited performance means you hit limits elsewhere. So your compiler may try to reduce latency now, which it wouldn't have done without OoO execution.

  • then does learning about processor architecture will help me in my program building as you are saying compilers can do these jobs? – user117953 Mar 4 '17 at 17:31
  • 3
    If you want to develop software as a life long career the most important thing is continuous curiosity and continuous desire to learn about everything. Processor architecture is one thing. Just type "intel processor architecture" into Google and start reading. – gnasher729 Mar 4 '17 at 19:44

The cynical answer:they don't! Historically, it is Intel that studied hundreds of executables to find the common sequences of instructions, then made the processors execute these sequences before even having checked if the entire sequence is really there. The most time consuming part is the beginning of a function which prepare the stack frame for the local variables. For C and C++, the cleaning is done immediately after the "call" instruction, for pascal, it is done at the end of the function itself. Some compiler try to help the processor by alternating the registers instead of using EAX" systematically. Other tricks used by some compilers is to interleave floating point instructions with integer. The 8087 chip was an external chip, completely autonomous and all x86 processors still implement the internal floating point operation as if it was a separate chip. So, both integer and floating point progress in parallel.

A cleverly designed compiler can replace the costly conditional jump implied by "if" with arithmetic operation which don't require a change in the instruction pointer address. The cost of changing PC is dramatic : it force the processor to discard all the instruction in the pipeline, undo any change that were tentatively computed when executing in advance so many instructions that followed the branch.

The compiler, unfortunately, can not guess the intent of programmers which try to cheat the operating system, for example to get their own thread the highest priority (damaging the delicate balance of fine tuned priority), or try to refresh a GUI item thousands of times per second (when the screen refresh is 60 Hz), send requests to a server a million time per second, or violate any other simple rule of good engineering.


Other answers are good, but let me just point out one thing - each human engineer can optimize their own "code", even if their code is sequencing in the processor.

So out-of-order execution is what the CPU engineer implements because that's what he/she is in charge of.

The operating system developer does what she/he can to overlap I/O, cache files, etc. because that's their responsibility.

Compiler and library developers - same idea.

Now take application programmers. You see it asserted all the time on stack exchange that compiler optimizers are so good (which is true) that you don't have to worry about optimizing application code (which is false). Rather, an application programmer should, first, go for ease of maintainability, but then, find the functions in need of memoizing, find the needless memory allocation, find the needless library calls that, deep down, shoot off I/O. These things and many more are the application programmer's responsibility. No amount of optimization by the CPU, by the OS, by the libraries, or by compilers, will do them for you, and they save "yuge" multiples in performance. This is my go-to method.


(I don't know what dynamic execution means, but I am familiar with the out-of-order execution idea)

While the x86-64 instruction set is (nearly) the same between very recent processors and those from five years ago (perhaps with the exception of vector instruction extensions like AVX-512), the optimization rules have changed.

In other words, a machine code program (executable or library) generated and optimized for an i3770 from five years ago won't be the best optimized code for a recent AMD Ryzen (but it will be able to run on that Ryzen 1800x without recompilation, since using the same instruction set). So machine code optimized for Ryzen is not the best for i3770 and vice versa.

But compilers know that, and they can generate optimized code for some particular CPU brand. They then would generate code tailored to specific features of your particular processor (e.g. size and organization of CPU cache, configuration & behavior of its branch predictor, out-of-order execution & pipelines, etc etc etc...)

So you need to update your compiler (it could be important to get a recent version) and pass appropriate options to it.

For GCC, read its optimizing options section, and also the x86 options section.

I recommend (for straight, not cross, compilation) compiling with gcc -Wall -O2 -march=native on the same machine as the one running your code. If you want to compile into some binary able to run on several brands you need to be more careful.

BTW, some compilers (including GCC...) provide extra builtins (and even x86-specific builtins) to take advantage of processor features not visible to C code; for example __builtin_prefetch (for explicit CPU cache prefetching), but see this which suggests to avoid using them in general (because compilers are often - but not always - optimizing well enough).

In practice, recent compilers do a pretty good job on optimizations, so I don't think it is worth the trouble to hand-tune code to optimize it even more, but YMMV. It might worth to recompile your (or some free software) source code when upgrading your processor, but you'll usually only win a few percents of performance in doing so.

(IMHO, cache locality matters a lot more, since a major -L3- cache miss is taking as much time as executing more than a hundred of machine instructions; e.g. prefer vectors to linked lists)

  • then does learning about processor architecture will help me in my program building as you are saying compilers can do these jobs? – user117953 Mar 4 '17 at 17:34
  • It could help to read something about recent processor architecture (e.g. understand the role of CPU cache). But you probably don't need to dive into the specific details of your processor brand. – Basile Starynkevitch Mar 4 '17 at 17:54
  • Of course, if you need to get the last % of efficiency from your particular processor, you need to know very well its internals. Processor makers (Intel & AMD) are publishing optimization guides. But most people (including me) don't care, and trust their recent compiler to optimize well enough. – Basile Starynkevitch Mar 4 '17 at 17:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.