Scatter Matrix Blocks of Different Sizes using MPI

Question

(Suppose all the matrices are stored in row-major order.) An example that illustrate the problem is to distribute a 10x10 matrix over a 3x3 grid, so that the size of the sub

Answer 1

Not sure if that applies to you, but it helped me in the past so it might be usefull to others.

My answer applies in the context of parallele IO. The thing is that, if you know your access are not overlapping, you can successfully write/read even with variable sizes by using MPI_COMM_SELF

A piece of code I use every day contains:

MPI_File fh;
MPI_File_open(MPI_COMM_SELF, path.c_str(), MPI_MODE_CREATE|MPI_MODE_WRONLY, MPI_INFO_NULL, &fh);

// Lot of computation to get the size right

MPI_Datatype filetype;
MPI_Type_create_subarray(gsizes.size(), &gsizes[0], &lsizes[0], &offset[0], MPI_ORDER_C, MPI_FLOAT, &filetype);
MPI_Type_commit(&filetype);

MPI_File_set_view(fh, 0, MPI_FLOAT, filetype, "native", MPI_INFO_NULL);
MPI_File_write(fh, &block->field[0], block->field.size(), MPI_FLOAT, MPI_STATUS_IGNORE);
MPI_File_close(&fh);