MPI_Allgather with 2D arrays

问题

I am trying to calculate the position of some bodies that is based in their previous positions. So in every k loop I need every C array to be updated with the new coordinates (x,y,z) of the bodies that is calculated and stored in the Cw arrays. I tried MPI_Allgather but I can't find the right syntax to achieve it.

I've checked the output with the serial version of the problem for k=1 and the values of F,V and Cw arrays are right so the only problem is the MPI_Allgather. The dt variable for now is equal to 1 for simplicity. I've tried this but none of ther work. The first one updates only the first row C array right from the Cw and 2 more values but in wrong places and the rest is the same as when is filled in the beginning. The second gives Segmentation fault

MPI_Allgather(&(Cw[0][0]),length*3,MPI_FLOAT,&(C[0][0]),length*3,MPI_FLOAT,MPI_COMM_WORLD);

MPI_Allgather(Cw,length*3,MPI_FLOAT,C,length*3,MPI_FLOAT,MPI_COMM_WORLD);

Here is the code

float **C,**Cw;

C=malloc(N*sizeof(float*));
Cw=malloc(length*sizeof(float*));

for(i=0;i<length;i++)
{
    Cw[i]=malloc(3*sizeof(float));
}

for(i=0;i<N;i++)
{
    C[i]=malloc(3*sizeof(float));
}


for(k=0;k<loop;k++)
{
    for(i=start;i<=end;i++)                      
    {
        for(j=0;j<N;j++)                        
        {
            if(i!=j)
            {
                dx=C[j][0]-C[i][0];
                dy=C[j][1]-C[i][1];
                dz=C[j][2]-C[i][2];

                d=sqrt(pow(dx,2)+pow(dy,2)+pow(dz,2));

                F[i-start][0] -= G*M[i]*M[j]/pow(d,3)*dx;
                F[i-start][1] -= G*M[i]*M[j]/pow(d,3)*dy;
                F[i-start][2] -= G*M[i]*M[j]/pow(d,3)*dz;
            }
        }
    }

    for(i=0;i<length;i++)
    {
        for(j=0;j<3;j++)
        {
            a=F[i][j]/M[i+start];                 // α=F/m
            V[i][j]=V[i][j]+a*1;                 // V(n+1)=Vn+α*Δt
            Cw[i][j]=C[i+start][j]+V[i][j]*1;    // R(n+1)=Rn+Vn*Δt
        }
    }

    // where MPI_Allgather takes place
}

The output that I expect is this provided by the serial program https://drive.google.com/open?id=1fwLu8Jk3JEorFTvNJyOtti3K_zIw0ncw

The mpi version with this code

MPI_Allgather(&(Cw[0][0]),length*3,MPI_FLOAT,&(C[0][0]),length*3,MPI_FLOAT,MPI_COMM_WORLD);

gives this output https://drive.google.com/open?id=14cEFFRvNGUN_RK3u8Z31iRDtiTJs6_8I

回答1:

Once again this pointers-to-pointers nonsense (for scicomp): you are passing the pointers to the rows via MPI, not the contents.

I suggest that you avoid to allocate an array as N independent rows, as in:

float **C,**Cw;

// I guess that you missed to paste these two lines
C  = malloc( N * sizeof(float*) );    
Cw = malloc( length * sizeof(float*) );

for(i=0;i<length;i++)
{
    Cw[i]=malloc(3*sizeof(float));
}

for(i=0;i<N;i++)
{
    C[i]=malloc(3*sizeof(float));
}

The arrays allocated in this way aren't linear in memory, and cannot be used by the MPI functions globally. When you pass C, you are passing just the array of pointers. Passing &C[0][0], you are passing the array of the first 3 elements, but the other 3-elements-arrays aren't contiguous in memory, since they've been allocated independently. So a segfault is the best that you can achieve, being random results, the worst.

It's correct to allocate the needed memory in one single block, that MPI function can handle properly:

float *C,*Cw;
C  = malloc( N * 3 * sizeof(float) );    
Cw = malloc( length * 3 * sizeof(float) );

and replace any memory access as

            dx=C[j][0]-C[i][0];
            dy=C[j][1]-C[i][1];
            dz=C[j][2]-C[i][2];

with

            dx=C[3*j+0]-C[3*i+0];
            dy=C[3*j+1]-C[3*i+1];
            dz=C[3*j+2]-C[3*i+2];

In this way, passing C will result in passing the array contents correctly.

来源：https://stackoverflow.com/questions/58364672/mpi-allgather-with-2d-arrays

标签

arrays

mpi