how to delete the duplicate lines in file except the first matched line

Question

In the following configuration file

/etc/fine-tune.conf

We have duplicate lines as

clean_history_in_os=true

Answer 1

You can use this awk to delete all matching lines except the first one:

awk '!(/clean_history_in_os=true/ && n++)' file

To save file in place you can use this gnu awk command:

awk -i inplace '!(/clean_history_in_os=true/ && n++)' file

otherwise use temporary file as:

awk '!(/clean_history_in_os=true/ && n++)' file > $$.tmp && mv $$.tmp file

---

Here is one sed solution to do the same:

sed -i -n '0,/clean_history_in_os=true/p;/clean_history_in_os=true/!p' file