How can I exclude directories from grep -R?

Question

I want to traverse all subdirectories, except the \"node_modules\" directory.

Answer 1

find . ! -name "node_modules" -type d 

Answer 2

If you are grepping for code in a git repository and node_modules is in your .gitignore, you can use git grep. git grep searches the tracked files in the working tree, ignoring everything from .gitignore

git grep "STUFF"

Answer 3

Many correct answers have been given here, but I'm adding this one to emphasize one point which caused some rushed attempts to fail before: exclude-dir takes a pattern, not a path to a directory.

Say your search is:

grep -r myobject

And you notice that your output is cluttered with results from the src/other/objects-folder. This command will not give you the intended result:

grep -r myobject --exclude-dir=src/other/objects-folder

And you may wonder why exclude-dir isn't working! To actually exclude results from the objects-folder, simply do this:

grep -r myobject --exclude-dir=objects-folder

In other words, just use the folder name, not the path. Obvious once you know it.

From the man page:

--exclude-dir=GLOB
Skip any command-line directory with a name suffix that matches the pattern GLOB. When searching recursively, skip any subdirectory whose base name matches GLOB. Ignore any redundant trailing slashes in GLOB.

Answer 4

This syntax

--exclude-dir={dir1,dir2}

is expanded by the shell (e.g. Bash), not by grep, into this:

--exclude-dir=dir1 --exclude-dir=dir2

Quoting will prevent the shell from expanding it, so this won't work:

--exclude-dir='{dir1,dir2}'    <-- this won't work

The patterns used with --exclude-dir are the same kind of patterns described in the man page for the --exclude option:

--exclude=GLOB
    Skip files whose base name matches GLOB (using wildcard matching).
    A file-name glob can use *, ?, and [...]  as wildcards, and \ to
    quote a wildcard or backslash character literally.

The shell will generally try to expand such a pattern itself, so to avoid this, you should quote it:

--exclude-dir='dir?'

You can use the curly braces and quoted exclude patterns together like this:

--exclude-dir={'dir?','dir??'}

A pattern can span multiple path segments:

--exclude-dir='some*/?lse'

This would exclude a directory like topdir/something/else.

Answer 5

Recent versions of GNU Grep (>= 2.5.2) provide:

--exclude-dir=dir

which excludes directories matching the pattern dir from recursive directory searches.

So you can do:

grep -R --exclude-dir=node_modules 'some pattern' /path/to/search

For a bit more information regarding syntax and usage see

• The GNU man page for File and Directory Selection
• A related StackOverflow answer Use grep --exclude/--include syntax to not grep through certain files

For older GNU Greps and POSIX Grep, use find as suggested in other answers.

Or just use ack (Edit: or The Silver Searcher) and be done with it!

Answer 6

Frequently use this:

grep can be used in conjunction with -r (recursive), i (ignore case) and -o (prints only matching part of lines). To exclude files use --exclude and to exclude directories use --exclude-dir.

Putting it together you end up with something like:

grep -rio --exclude={filenames comma separated} \
--exclude-dir={directory names comma separated} <search term> <location>

Describing it makes it sound far more complicated than it actually is. Easier to illustrate with a simple example.

Example:

Suppose I am searching for current project for all places where I explicitly set the string value debugger during a debugging session, and now wish to review / remove.

I write a script called findDebugger.sh and use grep to find all occurrences. However:

For file exclusions - I wish to ensure that .eslintrc is ignored (this actually has a linting rule about debugger so should be excluded). Likewise, I don't want my own script to be referenced in any results.

For directory exclusions - I wish to exclude node_modules as it contains lots of libraries that do reference debugger and I am not interested in those results. Also I just wish to omit .idea and .git hidden directories because I don't care about those search locations either, and wish to keep the search performant.

So here is the result - I create a script called findDebugger.sh with:

#!/usr/bin/env bash
grep -rio --exclude={.eslintrc,findDebugger.sh} \
--exclude-dir={node_modules,.idea,.git} debugger .