Parallel Make Output

Question

When running a CMake generated Makefile with multiple processes (make -jN), the output often gets messed up like this:

[  8%] [  8%] [  9%] Building CXX object App/CMakeFiles/App.dir/src/File1.cpp.o
Building CXX object App/CMakeFiles/App.dir/src/File2.cpp.o
Building CXX object App/CMakeFiles/App.dir/src/File3.cpp.o

I'm not sure, but I think this behavior is also there for Makefiles not generated by CMake. I'd say it happens when multiple processes write to stdout at the same time.

I know I'm probably being pedantic, but is there any (simple) fix to this? ;)

Answer 1

If you're using GNU make, you can do it by redefining SHELL such that commands are wrapped by a trivial utility that ensures atomicity of information printed to standard output. Here's a more detailed description, including sample source for the wrapper utility.

Answer 2

I tried to get the CMake people to fix this, but apparently they don't want to. See http://www.cmake.org/Bug/view.php?id=7062.

Answer 3

The specific CMake bug related to interleaved make output using -jN with N>1 is CMake bug 0012991: "Parallel build output mess". It is still open in the "backlog" state waiting to be fixed.

This bug is actually annoying enough that it's a strong reason to switch to Ninja instead of make. Plus the fact that Ninja is faster than make. Ninja also uses an appropriate number of parallel jobs based on the number of CPU cores present. Also cool is how Ninja is, by default, very quiet: all progress happens on a single line in the terminal unless the build process emits messages or a build step fails. If a build step fails, Ninja prints the full command line that invoked it and displays the output. It's really nice since it makes any warning or error messages stand out. Although currently there is no colored terminal output: that would be a nice improvement but for me the advantages of Ninja over make are tremendous.

Answer 4

Sun's (now Oracle's) dmake available on Linux and Solaris takes care of that. See here and there.

Answer 5

Here is a simple working example of using a wrapper for Make. I'm not sure if I'd encourage it's use, but it's an idea.

# Makefile
SHELL = /tmp/test/wrapper

test: test1 test2

test1:
    $(eval export TARGET=$@)
    env

test2:
    $(eval export TARGET=$@)
    env

and this:

#!/usr/bin/env bash
# wrapper
bash $@ | sed -e "s/^/${TARGET} /"