Find unique lines

后端 未结 11 1910
情书的邮戳
情书的邮戳 2020-12-23 10:46

How can I find the unique lines and remove all duplicates from a file? My input file is

1
1
2
3
5
5
7
7

I would like the result to be:

11条回答
  •  难免孤独
    2020-12-23 11:34

    While sort takes O(n log(n)) time, I prefer using

    awk '!seen[$0]++'
    

    awk '!seen[$0]++' is an abbreviation for awk '!seen[$0]++ {print}', print line(=$0) if seen[$0] is not zero. It take more space but only O(n) time.

提交回复
热议问题