Generate sample of 1,000,000 random permutations

Question

I am working with large number of integer permutations. The number of elements in each permutation is K. The element size is 1 byte. I need to generate N unique random permu

Answer 1

Given that 10! ~= 3e6 i.e. for K > ~15 if you shuffle a list of K items a million times using the proper Fischer-Yates or Knuth shuffle then you are very likely to get a unique shuffle every time.

If you can save all one million unique permutations in memory in a set data-structure then you can shuffle a list of K items and add them to the set until you have a million of them.

Here's some Python that also shows a measure of how good the shuffle is at generating unique perms for varying K's:

>>> from math import factorial
>>> from random import shuffle
>>> 
>>> n = 1000000
>>> for k in range(16, 9, -1):
    perms = set()
    perm = list(range(k))
    trials = 0
    while len(perms) < n:
        trials += 1
        for i in range(n - len(perms)):
            shuffle(perm)
            perms.add(tuple(perm))
    print('N=%i, K=%i, trials=%i, K!//N= %i' % (n, k, trials, factorial(k)//n))


N=1000000, K=16, trials=1, K!//N= 20922789
N=1000000, K=15, trials=1, K!//N= 1307674
N=1000000, K=14, trials=2, K!//N= 87178
N=1000000, K=13, trials=2, K!//N= 6227
N=1000000, K=12, trials=3, K!//N= 479
N=1000000, K=11, trials=5, K!//N= 39
N=1000000, K=10, trials=11, K!//N= 3
>>>