Nested Parallel.For() loops speed and performance

Question

I have a nested for loop. I have replaced the first For with a Parallel.For() and the speed of calculation increased.

My question is about replacing the

Answer 1

From "Too fine-grained, too coarse-grained" subsection, "Anti-patterns" section in "Patterns of parallel programming" book by .NET parallel computing team:

The answer is that the best balance is found through performance testing. If the overheads of parallelization are minimal as compared to the work being done, parallelize as much as possible: in this case, that would mean parallelizing both loops. If the overheads of parallelizing the inner loop would degrade performance on most systems, think twice before doing so, as it’ll likely be best only to parallelize the outer loop.

Take a look at that subsection, it is self-contained with detailed examples from parallel ray tracing application. And its suggestion of flattening the loops to have better degree of parallelism may be helpful for you too.