How do I ask OpenMP to create threads only once at each run of the program?

Question

I am trying to parallelize a large program that is written by a third-party. I cannot disclose the code, but I will try and give the closest example of what I wish to do. Ba

Answer 1

This can be done! The key here is to move the loop inside one single parallel section and make sure that whatever is used to determine whether to repeat or not, all threads will make exactly the same decision. I've used shared variables and do a synchronization just before the loop condition is checked.

So this code:

initialize();
while (some_condition) {
  #pragma omp parallel
  {
     some_parallel_work();
  }
}

can be transformed into something like this:

#pragma omp parallel
{
  #pragma omp single
  {
    initialize();  //if initialization cannot be parallelized
  }
  while (some_condition_using_shared_variable) {
    some_parallel_work();
    update_some_condition_using_shared_variable();
    #pragma omp flush
  }
}

The most important thing is to be sure that every thread makes the same decision at the same points in your code.

As a final thought, essentially what one is doing is trading the overhead for creating/destroying threads (every time a section of #pragma omp parallel begins/ends) into synchronization overhead for the decision making of the threads. I think synchronizing should be faster however there are some many parameters at play here that this may not always be.

Answer 2

OpenMP only creates worker thread at start. parallel pragma does not spawn thread. How do you determine the thread are spawned?