Randomly Pick Lines From a File Without Slurping It With Unix

前端 未结 10 937
忘了有多久
忘了有多久 2020-12-07 11:40

I have a 10^7 lines file, in which I want to choose 1/100 of lines randomly from the file. This is the AWK code I have, but it slurps all the file content before hand. My PC

10条回答
  •  攒了一身酷
    2020-12-07 11:53

    You used awk, but I don't know if it's required. If it's not, here's a trivial way to do w/ perl (and without loading the entire file into memory):

    cat your_file.txt | perl -n -e 'print if (rand() < .01)'
    

    (simpler form, from comments):

    perl -ne 'print if (rand() < .01)' your_file.txt 
    

提交回复
热议问题