Is MPI_Gather the best choice?

问题

There are 4 processes and one of them (0) is the master which has to build the matrix C as follow

-1  0  0 -1  0
 0 -1  0  0 -1
-1  1  1 -1  1
 1 -1  1  1 -1
-1  2  2 -1  2
 2 -1  2  2 -1
-1  3  3 -1  3
 3 -1  3  3 -1

To do so, the matrix is declared as REAL, DIMENSION(:,:), ALLOCATABLE :: C and allocated with

IF (myid == 0) THEN
        ALLOCATE(C(2*nprocs,-2:+2))
END IF

where nprocs is the number of processes. Process 0 also sets C = -1. For the communications I first tried with

CALL MPI_GATHER((/0.0+myid,0.0+myid/),&
              & 2,MPI_REAL,&
              & C(:,0),&
              & 2,MPI_REAL,&
              & 0,MPI_COMM_WORLD,ieri)

to fill up the central column, and this worked. Then I tried with

CALL MPI_GATHER((/myid, myid, myid, myid/),&
              & 4,MPI_REAL,&
              & (/C(1:2*nprocs:2,-1),C(2:2*nprocs:2,-2),C(1:2*nprocs:2,+2),C(2:2*nprocs:2,+1)/),&
              & 4,MPI_REAL,&
              & 0,MPI_COMM_WORLD,ierr)

to fill the other columns, but it didn't work, giving errors like the following

Fortran runtime error: Index '1' of dimension 1 of array 'c' outside of expected range (140735073734712:140735073734712).

To understand why, I tried to fill the first column alone with the call

CALL MPI_GATHER((/0.0-myid/),&
              & 1,MPI_REAL,&
              & C(1:2*nprocs:2,-2),&
              & 1,MPI_REAL,&
              & 0,MPI_COMM_WORLD,ierr)

but the same happened, more or less.

I solved the problem by allocating C for all the processes (i.e. regardless of the process id). Why does this make the call work?

After this I did a little change (before trying again to fill all the columns at once) simply putting the receive buffer in (/.../)

CALL MPI_GATHER((/0.0-myid/),&
              & 1,MPI_REAL,&
              & (/C(1:2*nprocs:2,-2)/),&
              & 1,MPI_REAL,&
              & 0,MPI_COMM_WORLD,ieri)

but this makes the call ineffective (no errors, but not even one element in C changed).

Hope someone can explain to me

what's wrong with the constructor (/.../) in the receive buffer?
why the receive buffer has to be allocated in the non-root processes?
it is necessary to use mpi_gatherv to accomplish the task?
is there a better way to build up such a matrix?

EDIT Is it possible to use MPI derived data types to build the matrix?

回答1:

First do use use mpi instead of include mpif.h if you are not doing that already. Some of these errors might be found by this.

You cannot use an array constructor as a receive buffer. Why? The array created by a constructor is an expression. You cannot use it where a variable is required.

The same way you cannot pass 1+1 to a subroutine which changes is argument. 1+1 is an expression and you need a variable if it is to be changed.

Secondly, every array into which you write or from which you read must be allocated. In MPI_Gather the receive buffer is ignored for all nonroot processes. BUT when you make a subarray from an array like C(1:2*nprocs:2,-2) from C, such an array must be allocated. This is a Fortran thing, not an MPI one.

If the number of elements received from each rank is the same you can use MPI_Gather, you don' need MPI_Gatherv.

You may consider just receiving the data into a 1D buffer and reorder them as necessary. Another option is to decompose it along the last dimension instead.

来源：https://stackoverflow.com/questions/35944584/is-mpi-gather-the-best-choice

标签

fortran

mpi

gfortran