How to delete duplicate lines in a file without sorting it in Unix?

后端 未结 9 1617
Happy的楠姐
Happy的楠姐 2020-11-22 17:26

Is there a way to delete duplicate lines in a file in Unix?

I can do it with sort -u and uniq commands, but I want to use sed

9条回答
  •  萌比男神i
    2020-11-22 17:48

    From http://sed.sourceforge.net/sed1line.txt: (Please don't ask me how this works ;-) )

     # delete duplicate, consecutive lines from a file (emulates "uniq").
     # First line in a set of duplicate lines is kept, rest are deleted.
     sed '$!N; /^\(.*\)\n\1$/!P; D'
    
     # delete duplicate, nonconsecutive lines from a file. Beware not to
     # overflow the buffer size of the hold space, or else use GNU sed.
     sed -n 'G; s/\n/&&/; /^\([ -~]*\n\).*\n\1/d; s/\n//; h; P'
    

提交回复
热议问题