git-filter-branch to remove strings, but where strings contain $ ' \ and other characters

Question

I\'m trying to rewrite history, using:

git filter-branch --tree-filter \'git ls-files -z \"*.php\" |xargs -0 perl -p -i -e \"s#(PASSWORD1|PASSWORD2|PASSWORD3)#

Answer 1

try the BFG instead of git filter-branch...

You can use a much more friendly substitution format if you use The BFG rather than git-filter-branch. Create a passwords.txt file, with one password per line like this:

PASSWORD1==>xXxXx      # Replace literal string 'PASSWORD1' with 'xXxXx'
ezxcdf\fr$sdd%==>xXxXx # ...all text is matched as a *literal* string by default

Then run the BFG with this command:

$ java -jar bfg.jar -fi '*.php' --replace-text passwords.txt  my-repo.git

Your entire repository history will be scanned, and all .php files (under 1MB in size) will have the substitutions performed: any matching string (that isn't in your latest commit) will be replaced.

...no escaping needed

Note that the only bit of parsing the BFG does with here with the substitution file is to split on the '==>' string - which probably isn't in your passwords - and all text is interpreted literally by default.

If you want to be even more concise, you can drop the '==>' and everything that comes after it on each line (ie, just have a file of passwords) and The BFG will replace each password with the string '***REMOVED***' by default.

The BFG is typically hundreds of times faster than running git-filter-branch on a big repo and the options are tailored around these two common use-cases:

• Removing Crazy Big Files
• Removing Passwords, Credentials & other Private data

Full disclosure: I'm the author of the BFG Repo-Cleaner.