Escape dollar sign in regexp for sed

Question

I will introduce what my question is about before actually asking - feel free to skip this section!

Some background info about my setup

Answer 1

The correct way to escape a dollar sign in regular expressions for sed is double-backslash. Then, for creating the escaped version in the output, we need some additional slashes:

cat filenames.txt | sed "s/\\$/\\\\$/g" > escaped-filenames.txt

Yep, that's four backslashes in a row. This creates the required changes: a filename like bla$1$2.class would then change to bla\$1\$2.class. This I can then insert into the full pipeline:

for i in $(diff -r old new 2>/dev/null | grep "Only in old" | cut -d "/" -f 3- | sed "s/: /\//g" | sed "s/\\$/\\\\$/g"; do echo "rm -f $i" >> REMOVEOLDFILES.sh; done

Alternative to solve the background problem

chepner posted an alternative to solve the backround problem by simply adding single-quotes around the filenames for the output. This way, the $-signs are not read as variables by bash when executing the script and the files are also properly removed:

for i in $(diff -r old new 2>/dev/null | grep "Only in old" | cut -d "/" -f 3- | sed "s/: /\//g"); do echo "rm -f '$i'" >> REMOVEOLDFILES.sh; done

(note the changed echo "rm -f '$i'" in that line)

Answer 2

There are other problems with your script, but file names containing $ are not a problem if you properly quote the argument to rm in the resulting script.

echo "rm -f '$i'" >> REMOVEOLDFILES.sh

or using printf, which makes quoting a little nicer and is more portable:

printf "rm -f '%s'" "$i" >> REMOVEOLDFILES.sh

(Note that I'm addressing the real problem, not necessarily the question you asked.)

Answer 3

There is already a nice answer directly in the edited question that helped me a lot - thank you!

I just want to add a bit of curious behavior that I stumbled across: matching against a dollar sign at the end of lines (e.g. when modifying PS1 in your .bashrc file). As a workaround, I match for additional whitespace.

$ DOLLAR_TERMINATED="123456 $"
$ echo "${DOLLAR_TERMINATED}" | sed -e "s/ \\$/END/"
123456END
$ echo "${DOLLAR_TERMINATED}" | sed -e "s/ \\$$/END/"
sed: -e expression #1, char 13: Invalid back reference
$ echo "${DOLLAR_TERMINATED}" | sed -e "s/ \\$\s*$/END/"
123456END

Explanation to the above, line by line:

• Defining DOLLAR_TERMINATED - I want to replace the dollar sign at the end of DOLLAR_TERMINATED with "END"
• It works if I don't check for the line ending
• It won't work if I match for the line ending as well (adding one more $ on the left side)
• It works if I additionally match for (non-present) whitespace

(My sed version is 4.2.2 from February 2016, bash is version 4.3.48(1)-release (x86_64-pc-linux-gnu), in case that makes any difference)