Shell: Find Matching Lines Across Many Files

Question

I am trying to use a shell script (well a \"one liner\") to find any common lines between around 50 files. Edit: Note I am looking for a line (lines) that a

Answer 1

When I first read this I thought you were trying to find 'any common lines'. I took this as meaning "find duplicate lines". If this is the case, the following should suffice:

sort *.sp | uniq -d

---

Upon re-reading your question, it seems that you are actually trying to find lines that 'appear in all the files'. If this is the case, you will need to know the number of files in your directory:

find . -type f -name "*.sp" | wc -l

If this returns the number 50, you can then use awk like this:

WHINY_USERS=1 awk '{ array[$0]++ } END { for (i in array) if (array[i] == 50) print i }' *.sp

---

You can consolidate this process and write a one-liner like this:

WHINY_USERS=1 awk -v find=$(find . -type f -name "*.sp" | wc -l) '{ array[$0]++ } END { for (i in array) if (array[i] == find) print i }' *.sp