When I (re-)build large systems on a desktop/laptop computer, I tell make to use more than one thread to speed up the compilation speed, like this:

$ make -j$[ $K * $C ]

Where $C is supposed to indicate the number of cores (which we can assume to be a number with one digit) the machine has, while $K is something I vary from 2 to 4, depending on my mood.

So, for example, I might say make -j12 if I have 4 cores, indicating to make to use up to 12 threads.

My rationale is, that if I only use $C threads, cores will be idle while processes are busy fetching data from the drives. But if I do not limit the number of threads (i.e. make -j) I run the risk to waste time switching contexts, run out of memory, or worse. Let's assume the machine has $M gigs of memory (where $M is in the order of 10).

So I was wondering if there is an established strategy to choose the most efficient number of threads to run.

  • In many cases, the correct answer for number of threads is going to be the number of cores. But the only way to know for sure is to run some tests, varying the number of threads until you find the sweet spot. – Robert Harvey Jul 12 '12 at 17:27
  • @RobertHarvey: Yes, I'll probably go and have a shell script compile with all sorts of settings over night, but I thought I'd ask if there is some knowledge about this out there. – bitmask Jul 12 '12 at 17:33
  • 4
    many people also suggest $cores+1, so 1 compiler process reads from disk while 4 compile. A generic suggestion is hard, also depends on the code base (C++ template overuse vs. small compilation units with a few C functions), compiler chain (precompiled headers etc?) and the build structure (is it linking just one big thing in the end or multiple smaller things in between) – johannes Jul 12 '12 at 17:40
  • 1
    If you're seriously looking for performance, I'd suggest looking into setting up a RAM disk or some other method of alleviating your I/O. I don't think CPU utilization is your hot spot. – TMN Jul 12 '12 at 19:28
  • @TMN: How a RAM disk help? Linux is pretty good at caching stuff (you do mean the header files, right?), not to mention the drive cache. I would have to load everything into the shm first, either manually or by changing the build script (which would be utter overkill). – bitmask Jul 12 '12 at 19:37

I ran a series of tests, building llvm (in Debug+Asserts mode) on a machine with two cores and 8 GB of RAM:

compiling llvm time depending on number of jobs

Oddly enough, it seems to climb until 10 and then suddenly drops below the time it takes to build with two jobs (one jobs takes about the double time, not included in the graph).

The minimum seems to be 7*$cores in this case.

  • 1
    +1 for actual testing and not speculating. – Martin Wickman Jul 15 '12 at 8:08

I'm running Gentoo Linux (source-based distribution) and from my experience i can say that (with more or less recent hardware) n*2 + x is the best value. Let me explain this:

  • n*2: Even slower CPU's have enough power to run 2 tasks at a time. most compile tasks are completed very fast.
  • +x this number depends on your system (mainly memory and disk). If you have enough RAM and a fast disk, set x=n. However, this depends on the source code (Open Office, i'm looking at you!) and the used language (compiling C/C++ is very memory intensive).

However, you have to run some tests with some -j values to get the best number. Also, try to parallelize other steps of the build process: unpacking, running configure and so on.

  • I'm mostly concerned with C++ at the moment, and my disks are not the fastest, I guess. – bitmask Jul 13 '12 at 8:21
  • Then start with n*1.5 and increase it until the compile times stops decreasing (make sure you clean the disk cache/compile cache every time). Also, think of using ccache (ccache.samba.org) to speed up the compilation. – ercpe Jul 13 '12 at 9:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.