Grepping a huge file (80GB) any way to speed it up?

Question

 grep -i -A 5 -B 5 \'db_pd.Clients\'  eightygigsfile.sql

This has been running for an hour on a fairly powerful linux server which is otherwise not

Answer 1

If you have a multicore CPU, I would really recommend GNU parallel. To grep a big file in parallel use:

< eightygigsfile.sql parallel --pipe grep -i -C 5 'db_pd.Clients'

Depending on your disks and CPUs it may be faster to read larger blocks:

< eightygigsfile.sql parallel --pipe --block 10M grep -i -C 5 'db_pd.Clients'

It's not entirely clear from you question, but other options for grep include:

• Dropping the -i flag.
• Using the -F flag for a fixed string
• Disabling NLS with LANG=C
• Setting a max number of matches with the -m flag.