Advanced `uniq` with “unique part regex”

Question

uniq is a tool that enables once to filter lines in a file such that only unique lines are shown. uniq has some support to specify when two lines a

Answer 1

Here's a simple Perl script that will do the work:

#!/usr/bin/env perl
use strict;
use warnings;

my $re = qr($ARGV[0]);

my %matches;
while() {
    next if $_ !~ $re;
    print if !$matches{$1};
    $matches{$1} = 1;
}

Usage:

$ ./uniq.pl '(!\w+!)' < file.dat
foo!bar!baz
!baz!quix

Here, I've used $1 to match on the first extracted group, but you can replace it with $& to use the whole pattern match.
This script will filter out lines that don't match the regex, but you can adjust it if you need a different behavior.