perl to delete consecutive duplicate lines

后端 未结 4 511
感情败类
感情败类 2021-01-21 02:42

I want to delete the consecutive duplicate lines. i.e. for example

**test.txt**
car
speed is good
bike 
slower than car
plane
super fast
super fast
bullet train          


        
4条回答
  •  不要未来只要你来
    2021-01-21 03:16

    I also wanted to keep track of how many duplicates were suppressed and only skip consecutive duplicates.

    While this is not exactly what the OP asked, it is a variant which others may find useful:

    perl -ne 'if (defined($pr) && ($_ eq $pr)) {$cnt++;} else {print "... (+$cnt)\n" if ($cnt); print; $cnt=0; $pr=$_;}'
    

    It produced something like this with my data (a database restore log):

    COPY 9
    COPY 0
    ... (+2)
    COPY 5
    COPY 0
    ... (+1)
    COPY 24
    ALTER TABLE
    ... (+23)
    CREATE INDEX
    ... (+73)
    

提交回复
热议问题