mpiexec and python mpi4py gives rank 0 and size 1

问题

I have a problem with running a python Hello World mpi4py code on a virtual machine.

The hello.py code is:

#!/usr/bin/python
#hello.py
from mpi4py import MPI

comm = MPI.COMM_WORLD

size = comm.Get_size()
rank = comm.Get_rank()

print "hello world from process ", rank,"of", size

I've tried to run it using mpiexec and mpirun, but it is not running well. The output:

$ mpirun -c 4 python hello.py 
hello world from process  0 of 1
hello world from process  0 of 1
hello world from process  0 of 1
hello world from process  0 of 1

And from mpiexec:

$ mpiexec -n 4 python hello.py 
hello world from process  0 of 1
hello world from process  0 of 1
hello world from process  0 of 1
hello world from process  0 of 1

They seem not getting rank and size of comm. What can cause this? How to solve it?

mpiexec --version
mpiexec (OpenRTE) 1.6.5

mpirun --version
mpirun (Open MPI) 1.6.5

The system is Ubuntu 14.04 on the Virtal Machine.

Any ideas why? Thanks!

回答1:

I had the same issue when running the python module emcee. It would give me an error:

"ValueError: Tried to create an MPI pool, but there was only one MPI process available. 
Need at least two."

The solution I found for my particular cluster was to use a different MPI. My code worked with intel-mpi and mpich2 but not openmpi. For this system, all I had to do was switch the MPI. In my PBS script I used module load mpich2 instead of module load openmpi. In this case mpiexec and mpirun worked correctly.

回答2:

As suggested above and in this question for C, this has to do with having mpirun coming from a different MPI than mpi4py was linked against.

In my case, and I suspect the same is true for many other Python users, this came from having originally installed mpi4py through conda, which pulled a non-system version of MPI into my conda -- i.e. which mpirun gave a path in my conda environment.

To solve the problem, I ran conda remove mpi4py and then pip install mpi4py, which seemed to rebuild mpi4py against the MPI in the conda environment and solved the issue.

来源：https://stackoverflow.com/questions/29264640/mpiexec-and-python-mpi4py-gives-rank-0-and-size-1

标签

python

mpi

mpi4py

mpiexec