OpenMP parallelizing matrix multiplication by a triple for loop (performance issue)

Question

I\'m writing a program for matrix multiplication with OpenMP, that, for cache convenience, implements the multiplication A x B(transpose) rows X rows instead of the classic A x

Answer 1

You could probably have some dependencies in the data when you parallelize the outer loop and compiler is not able to figure it out and adds additional locks.

Most probably it decides that different outer loop iterations could write into the same (C+(i*Nu+j)) and it adds access locks to protect it.

Compiler could probably figure out that there are no dependencies if you'll parallelize the 2nd loop. But figuring out that there are no dependencies parallelizing the outer loop is not so trivial for a compiler.

UPDATE

Some performance measurements.

Hi again. It looks like 1000 double * and + is not enough to cover the cost of threads synchronization.

I've done few small tests and simple vector scalar multiplication is not effective with openmp unless the number of elements is less than ~10'000. Basically, larger your array is, more performance will you get from using openmp.

So parallelizing the most inner loop you'll have to separate task between different threads and gather data back 1'000'000 times.

PS. Try Intel ICC, it is kinda free to use for students and open source projects. I remember being using openmp for smaller that 10'000 elements arrays.

UPDATE 2: Reduction example

    double sum = 0.0;
    int k=0;
    double *al = A+i*Nu;
    double *bl = A+j*Nu;
    #pragma omp parallel for shared(al, bl) reduction(+:sum)
    for(k=0;k