mpiexec and python mpi4py gives rank 0 and size 1

Question

I have a problem with running a python Hello World mpi4py code on a virtual machine.

The hello.py code is:

#!/usr/bin/python
#hello.py
from mpi4py import         


        

Answer 1

I had the same issue when running the python module emcee. It would give me an error:

"ValueError: Tried to create an MPI pool, but there was only one MPI process available. 
Need at least two."

The solution I found for my particular cluster was to use a different MPI. My code worked with intel-mpi and mpich2 but not openmpi. For this system, all I had to do was switch the MPI. In my PBS script I used module load mpich2 instead of module load openmpi. In this case mpiexec and mpirun worked correctly.

Answer 2

As suggested above and in this question for C, this has to do with having mpirun coming from a different MPI than mpi4py was linked against.

In my case, and I suspect the same is true for many other Python users, this came from having originally installed mpi4py through conda, which pulled a non-system version of MPI into my conda -- i.e. which mpirun gave a path in my conda environment.

To solve the problem, I ran conda remove mpi4py and then pip install mpi4py, which seemed to rebuild mpi4py against the MPI in the conda environment and solved the issue.