Python multiprocessing: restrict number of cores used

Question

I want to know how to distribute N independent tasks to exactly M processors on a machine that has L cores, where L>M. I don\'t want to use all the processors because I sti

Answer 1

If you are on linux, use taskset when you launch the program

A child created via fork(2) inherits its parent’s CPU affinity mask. The affinity mask is preserved across an execve(2).

TASKSET(1)
Linux User’s Manual
TASKSET(1)

NAME taskset - retrieve or set a process’s CPU affinity

SYNOPSIS taskset [options] mask command [arg]... taskset [options] -p [mask] pid

DESCRIPTION taskset is used to set or retrieve the CPU affinity of a running process given its PID or to launch a new COMMAND with a given CPU affinity. CPU affinity is a scheduler property that "bonds" a process to a given set of CPUs on the system. The Linux scheduler will honor the given CPU affinity and the process will not run on any other CPUs. Note that the Linux scheduler also supports natural CPU affinity: the scheduler attempts to keep processes on the same CPU as long as practical for performance reasons. Therefore, forcing a specific CPU affinity is useful only in certain applications.

The CPU affinity is represented as a bitmask, with the lowest order bit corresponding to the first logical CPU and the highest order bit corresponding to the last logical CPU. Not all CPUs may exist on a given sys‐ tem but a mask may specify more CPUs than are present. A retrieved mask will reflect only the bits that cor‐ respond to CPUs physically on the system. If an invalid mask is given (i.e., one that corresponds to no valid CPUs on the current system) an error is returned. The masks are typically given in hexadecimal.