grep -vf too slow with large files

Question

I am trying filter data from data.txt using patterns stored in a file filter.txt. Like below,

grep -v -f filter.txt data.txt > op.txt

Th

Answer 1

Based on Inian's solution in the related post, this awk command should solve your issue:

awk 'FNR==NR {hash[$0]; next} !($0 in hash)' filter.txt data.txt > op.txt