How to remove trailing whitespace of all files recursively?

Question

How can you remove all of the trailing whitespace of an entire project? Starting at a root directory, and removing the trailing whitespace from all files in all folders.

Answer 1

Two alternative approaches which also work with DOS newlines (CR/LF) and do a pretty good job at avoiding binary files:

Generic solution which checks that the MIME type starts with text/:

while IFS= read -r -d '' -u 9
do
    if [[ "$(file -bs --mime-type -- "$REPLY")" = text/* ]]
    then
        sed -i 's/[ \t]\+\(\r\?\)$/\1/' -- "$REPLY"
    else
        echo "Skipping $REPLY" >&2
    fi
done 9< <(find . -type f -print0)

Git repository-specific solution by Mat which uses the -I option of git grep to skip files which Git considers to be binary:

git grep -I --name-only -z -e '' | xargs -0 sed -i 's/[ \t]\+\(\r\?\)$/\1/'

Answer 2

This worked for me in OSX 10.5 Leopard, which does not use GNU sed or xargs.

find dir -type f -print0 | xargs -0 sed -i.bak -E "s/[[:space:]]*$//"

Just be careful with this if you have files that need to be excluded (I did)!

You can use -prune to ignore certain directories or files. For Python files in a git repository, you could use something like:

find dir -not -path '.git' -iname '*.py'

Answer 3

Ack was made for this kind of task.

It works just like grep, but knows not to descend into places like .svn, .git, .cvs, etc.

ack --print0 -l '[ \t]+$' | xargs -0 -n1 perl -pi -e 's/[ \t]+$//'

Much easier than jumping through hoops with find/grep.

Ack is available via most package managers (as either ack or ack-grep).

It's just a Perl program, so it's also available in a single-file version that you can just download and run. See: Ack Install

Answer 4

Ruby:

irb
Dir['lib/**/*.rb'].each{|f| x = File.read(f); File.write(f, x.gsub(/[ \t]+$/,"")) }

Answer 5

Use:

find . -type f -print0 | xargs -0 perl -pi.bak -e 's/ +$//'

if you don't want the ".bak" files generated:

find . -type f -print0 | xargs -0 perl -pi -e 's/ +$//'

as a zsh user, you can omit the call to find, and instead use:

perl -pi -e 's/ +$//' **/*

Note: To prevent destroying .git directory, try adding: -not -iwholename '*.git*'.

Answer 6

Here is an OS X >= 10.6 Snow Leopard solution.

It Ignores .git and .svn folders and their contents. Also it won't leave a backup file.

export LC_CTYPE=C
export LANG=C
find . -not \( -name .svn -prune -o -name .git -prune \) -type f -print0 | xargs -0 sed -i '' -E "s/[[:space:]]*$//"