Compiling with g++ using multiple cores

Question

Quick question: what is the compiler flag to allow g++ to spawn multiple instances of itself in order to compile large projects quicker (for example 4 source files at a time

Answer 1

GNU parallel

I was making a synthetic compilation benchmark and couldn't be bothered to write a Makefile, so I used:

sudo apt-get install parallel
ls | grep -E '\.c$' | parallel -t --will-cite "gcc -c -o '{.}.o' '{}'"

Explanation:

• {.} takes the input argument and removes its extension
• -t prints out the commands being run to give us an idea of progress
• --will-cite removes the request to cite the software if you publish results using it...

parallel is so convenient that I could even do a timestamp check myself:

ls | grep -E '\.c$' | parallel -t --will-cite "\
  if ! [ -f '{.}.o' ] || [ '{}' -nt '{.}.o' ]; then
    gcc -c -o '{.}.o' '{}'
  fi
"

xargs -P can also run jobs in parallel, but it is a bit less convenient to do the extension manipulation or run multiple commands with it: Calling multiple commands through xargs

Parallel linking was asked at: Can gcc use multiple cores when linking?

TODO: I think I read somewhere that compilation can be reduced to matrix multiplication, so maybe it is also possible to speed up single file compilation for large files. But I can't find a reference now.

Tested in Ubuntu 18.10.