Why don't compilers inline everything? [closed]

Question

Sometimes compilers inline function calls. That means that they move the code of the called function into the calling function. This makes things slightly faster because there's no need to push and pop stuff on and off the call stack.

So my question is, why don't compilers inline everything? I assume it would make the executable notably faster.

The only reason I can think of is a significantly larger executable, but does it really matter these days with hundreds of GB of memory? Isn't the improved performance worth it?

Is there any other reason why compilers don't just inline all function calls?

Answer 1

First note that one major effect of inline is that it allows further optimizations to be made at the call site.

For your question: there are things which are difficult or even impossible to inline:

• dynamically linked libraries

• dynamically determined functions (dynamic dispatch, called through function pointers)

• recursive functions (tail recursion can)

• functions for which you don't have the code (but link time optimization allow this for some of them)

Then inlining has not only beneficial effects:

• bigger executable means more disk place and bigger load time

• bigger executable means increase of cache pressure (note that inlining small enough functions such as simple getters may decrease the executable size and the cache pressure)

And finally, for functions which takes a non trivial time to execute, the gain just doesn't worth the pain.

Answer 2

A major limitation is runtime polymorphism. If there is a dynamic dispatch happening when you write foo.bar() then it is impossible to inline the method call. This explains why compilers don't inline everything.

Recursive calls cannot be easily inlined either.

Cross module inlining is also difficult to perform for technical reasons (incremental recompilation would be impossible for ex)

However, compilers do inline a lot of things.

Answer 3

First, you cannot always inline, e.g. recursive functions might not be always inlinable (but a program containing a recursive definition of fact with just a printing of fact(8) could be inlined).

Then, inlining is not always beneficial. If the compiler inlines so much that the result code is big enough to have its hot parts not fitting in e.g. the L1 instruction cache, it might be much slower than the non-inlined version (which would easily fit the L1 cache)... Also, recent processors are very fast at executing a CALL machine instruction (at least to a known location, i.e. a direct call, not a call thru pointer).

At last, full inlining requires a whole program analysis. This might not be possible (or is too costly). With C or C++ compiled by GCC (and also with Clang/LLVM) you need to enable link-time optimization (by compiling and linking with e.g. g++ -flto -O2) and that takes quite a lot of compilation time.

Answer 4

Surprising though it may seem, inlining everything doesn't necessarily reduce execution time. The increased size of your code can make it difficult for the CPU to keep all your code in its cache at once. A cache miss on your code becomes more likely and a cache miss is expensive. This is made far worse if your potentially inlined functions are large.

I've had noticeable performance improvements from time to time by taking large chunks of code marked as 'inline' out of header files, put them into the source code, so the code is only in one place rather than at every call site. Then the CPU cache is used better and you also get better compile time...

Answer 5

Inlining everything would not mean just increased disk memory consumption but also increased internal memory consumption which is not that plentiful. Remember that code also relies in memory in code segment; if a function is called from 10000 places (say the ones from standard libraries in a fairly large project), then the code for that function occupies 10000 times more internal memory.

Another reason might be the JIT compilers; if everything is inline then there are not hot spots to be dynamically compiled.

Answer 6

One, there are simple examples where inlining everything will work out very badly. Consider this simple C code:

void f1 (void) { printf ("Hello, world\n"); }
void f2 (void) { f1 (); f1 (); f1 (); f1 (); }
void f3 (void) { f2 (); f2 (); f2 (); f2 (); }
...
void f99 (void) { f98 (); f98 (); f98 (); f98 (); }

Guess what inlining everything will do to you.

Next, you make the assumption that inlining will make things faster. That's the case sometimes, but not always. One reason is that code that fits into the instruction cache runs a lot faster. If I call a function from 10 places, I'll always run code that is in the instruction cache. If it is inlined, then the copies are all over the place and run a lot slower.

There are other problems: Inlining produces huge functions. Huge functions are a lot harder to optimise. I've got considerable gains in performance critical code by hiding functions into a separate file to prevent the compiler from inlining them. As a result, the generated code for these functions was much better when they were hidden.

BTW. I don't have "hundreds of GBs of memory". My works computer doesn't even have "hundreds of GBs of hard drive space". And if my application where "hundreds of GBs of memory", it would take 20 minutes just to load the application to memory.