Is there a way to know the order of execution for a reduction operator in OpenMP? In other words, I would like to know how the threads execute reduction operation- is it left to right? What happens when there are numbers that are not power of 2?
I think you'll find that OpenMP will only reduce on associative operations, such as + and * (or addition and multiplication if you prefer) which means that it can proceed oblivious to the order of evaluation of the component parts of the reduction expression across threads.
I strongly suggest that you proceed in the same way when using OpenMP, trying to either figure out or constrain the order of execution will at best turn your parallel program into a sequential one, at worst proceed to give you effectively random results.
I fail to understand your last sentence about numbers that are not powers of 2.
As @JimCownie has pointed out, operations such as + and * are not strictly associative on floating-point numbers. Construe my reference to associative operations in the first sentence to mean operations which are, when applied to real numbers, associative, but which, for reasons well understood by skilled practitioners of numerical computing on modern computers, fail to be associative when applied to floating-point numbers.
The order of reduction is not specified by the OpenMP standard.
The standard allows reductions on floating point numbers (using +, for instance), which is not associative. If you don't understand that you should go and read "What every computer scientist should know about floating-point arithmetic".
来源：https://stackoverflow.com/questions/16306440/order-of-execution-in-reduction-operation-in-openmp