Generating the same random numbers with threads in OMP

Question

I am attempting to multithread some code with OMP. Currently my sequentially version using rand() to generate a set of random numbers with a consistent seed so that they ret

Answer 1

Here is a block based approach that divides the problem space in N/BLOCK_SIZE blocks and reseeds the RNG with your randInit + block number for each block. This gives a reproducible output no matter the number of threads you have. It also generates the same initial N numbers for sequence of N + x. This as long as you keep the same BLOCK_SIZE.

A good block-size is probably something like your typical N / (max_num_procs * 2). But there is room for experimentation.

#include 
#include 
#include 

#define N_DEFAULT  48   //Default number of nodes 
#define BLOCK_SIZE 12   //BLOCK SIZE number of nodes per block.
                        //Changes this changes reseed frequencey, 
                        //.. and hence the generated sequence
#define randInit   42   //Had to be something.

int main(int argc , char* argv[])
{
    int N=N_DEFAULT;
    if (argc >1)
        N=atoi(argv[1]); 

    int rands[N];// keep our random numbers for sequential debug output
    int n=BLOCK_SIZE;
    int num_blocks=(N+BLOCK_SIZE-1)/ BLOCK_SIZE;  // ceil(N/BLOCK_SIZE)
    int nt=omp_get_max_threads();   


    printf("          N: %d\n",N);
    printf("     Blocks: %d, (size: %d)\n",num_blocks,n);
    printf("    threads: %d\n",nt);

    //Parallel random generation
    #pragma omp parallel for schedule(runtime) 
    for (int J=0;J N?N:start+n;

        int tid = omp_get_thread_num(); // Just for debug
        printf("thread %d: works on block %d (%d - %d )\n",tid,J,start,end);

        for (int j=start; j < end;j++)
        {           
            int t=rand_r(&block_seed); //rand_r provides thread safe (re-entrant rand)
            rands[j]=t;
        }
    }

    //Output for debug single thread
    for (int j=0; j < N;j++)
    {
        printf("%d : %d \n",j,rands[j]);
    }   
    return 0;
}

Output with different N, and number of threads shown below.

N: 24                                   N: 27
Blocks: 3, (size: 8)                    Blocks: 4, (size: 8)
threads: 4                              threads: 1
-------------------------------------|-------------------------------
thread 1: works on block 1 (8 - 16 )    thread 0: works on block 0 (0 - 8 )
thread 2: works on block 2 (16 - 24 )   thread 0: works on block 1 (8 - 16 )
thread 0: works on block 0 (0 - 8 )     thread 0: works on block 2 (16 - 24 )
                                        thread 0: works on block 3 (24 - 27 )
-------------------------------------|-------------------------------
0 : 681191333                           0 : 681191333
1 : 928546885                           1 : 928546885
2 : 1457394273                          2 : 1457394273
3 : 941445650                           3 : 941445650
4 : 2129613237                          4 : 2129613237
5 : 1661015563                          5 : 1661015563
6 : 2071432601                          6 : 2071432601
7 : 222443696                           7 : 222443696
8 : 1156886562                          8 : 1156886562
9 : 398918689                           9 : 398918689
10 : 170756699                         10 : 170756699
11 : 703115845                         11 : 703115845
12 : 1424182583                        12 : 1424182583
13 : 1516198481                        13 : 1516198481
14 : 1740837599                        14 : 1740837599
15 : 1148851528                        15 : 1148851528
16 : 1633630368                        16 : 1633630368
17 : 2015727614                        17 : 2015727614
18 : 1031602773                        18 : 1031602773
19 : 463737465                         19 : 463737465
20 : 720848057                         20 : 720848057
21 : 1369285272                        21 : 1369285272
22 : 1411290150                        22 : 1411290150
23 : 2074210785                        23 : 2074210785
-------------------------------------|-------------------------------
                                       24 : 2109326622
                                       25 : 1486099418
                                       26 : 1892448847