11

When I (re-)build large systems on a desktop/laptop computer, I tell make to use more than one thread to speed up the compilation speed, like this:

$ make -j$[ $K * $C ]

Where $C is supposed to indicate the number of cores (which we can assume to be a number with one digit) the machine has, while $K is something I vary from 2 to 4, depending on my mood.

So, for example, I might say make -j12 if I have 4 cores, indicating to make to use up to 12 threads.

My rationale is, that if I only use $C threads, cores will be idle while processes are busy fetching data from the drives. But if I do not limit the number of threads (i.e. make -j) I run the risk to waste time switching contexts, run out of memory, or worse. Let's assume the machine has $M gigs of memory (where $M is in the order of 10).

So I was wondering if there is an established strategy to choose the most efficient number of threads to run.

Improve this question
9
  • In many cases, the correct answer for number of threads is going to be the number of cores. But the only way to know for sure is to run some tests, varying the number of threads until you find the sweet spot.
    – Robert Harvey
    Jul 12, 2012 at 17:27
  • @RobertHarvey: Yes, I'll probably go and have a shell script compile with all sorts of settings over night, but I thought I'd ask if there is some knowledge about this out there.
    – bitmask
    Jul 12, 2012 at 17:33
  • 4
    many people also suggest $cores+1, so 1 compiler process reads from disk while 4 compile. A generic suggestion is hard, also depends on the code base (C++ template overuse vs. small compilation units with a few C functions), compiler chain (precompiled headers etc?) and the build structure (is it linking just one big thing in the end or multiple smaller things in between)
    – johannes
    Jul 12, 2012 at 17:40
  • 1
    If you're seriously looking for performance, I'd suggest looking into setting up a RAM disk or some other method of alleviating your I/O. I don't think CPU utilization is your hot spot.
    – TMN
    Jul 12, 2012 at 19:28
  • @TMN: How a RAM disk help? Linux is pretty good at caching stuff (you do mean the header files, right?), not to mention the drive cache. I would have to load everything into the shm first, either manually or by changing the build script (which would be utter overkill).
    – bitmask
    Jul 12, 2012 at 19:37

2 Answers 2

Reset to default
16

I ran a series of tests, building llvm (in Debug+Asserts mode) on a machine with two cores and 8 GB of RAM:

compiling llvm time depending on number of jobs

Oddly enough, it seems to climb until 10 and then suddenly drops below the time it takes to build with two jobs (one jobs takes about the double time, not included in the graph).

The minimum seems to be 7*$cores in this case.

Improve this answer
1
  • 3
    +1 for actual testing and not speculating.
    – Martin Wickman
    Jul 15, 2012 at 8:08
3

I'm running Gentoo Linux (source-based distribution) and from my experience i can say that (with more or less recent hardware) n*2 + x is the best value. Let me explain this:

  • n*2: Even slower CPU's have enough power to run 2 tasks at a time. most compile tasks are completed very fast.
  • +x this number depends on your system (mainly memory and disk). If you have enough RAM and a fast disk, set x=n. However, this depends on the source code (Open Office, i'm looking at you!) and the used language (compiling C/C++ is very memory intensive).

However, you have to run some tests with some -j values to get the best number. Also, try to parallelize other steps of the build process: unpacking, running configure and so on.

Improve this answer
2
  • I'm mostly concerned with C++ at the moment, and my disks are not the fastest, I guess.
    – bitmask
    Jul 13, 2012 at 8:21
  • Then start with n*1.5 and increase it until the compile times stops decreasing (make sure you clean the disk cache/compile cache every time). Also, think of using ccache (ccache.samba.org) to speed up the compilation.
    – ercpe
    Jul 13, 2012 at 9:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.