IPython.parallel not using multicore?

Question

I am experimenting with IPython.parallel and just want to launch several shell command on different engines.

I have the following Notebook:

Answer 1

Summary of the chat discussion:

CPU affinity is a mechanism for pinning a process to a particular CPU core, and the issue here is that sometimes importing numpy can end up pinning Python processes to CPU 0, as a result of linking against particular BLAS libraries. You can unpin all of your engines by running this cell:

%%px
import os
import psutil
from multiprocessing import cpu_count

p = psutil.Process(os.getpid())
p.set_cpu_affinity(range(cpu_count()))
print p.get_cpu_affinity()

Which uses multiprocessing.cpu_count to get the number of CPUs, and then associates every engine with all CPUs.

An IPython notebook exploring the issue.