bash - shuffle a file that is too large to fit in memory

Question

I\'ve got a file that\'s too large to fit in memory. shuf seems to run in RAM, and sort -R doesn\'t shuffle (identical lines end up next to each ot

Answer 1

Have a look at https://github.com/alexandres/terashuf . From page:

terashuf implements a quasi-shuffle algorithm for shuffling multi-terabyte text files using limited memory