How to escape history expansion exclamation mark ! inside a double quoted string?

Question

EDIT: the command substitution is not necessary for the surprising behavior, although it is the most common use case. The same question applies to just echo "'!b'"

b=a

# Enable history substitution.
# This option is on by default on interactive shells.
set -H

echo '!b'
# Output: '!b'
# OK. Escaped by single quotes.

echo $(echo '!b')
# Output: '!b'
# OK. Escaped by single quotes.

echo "$(echo '$b')"
# Output: '$b'
# OK. Escaped by single quotes.

echo "$(echo '!b')"
# Output: history expands
# BAD!! WHY??

• In the last example, what is the best way to escape the !?
• Why was it not escaped even if I used single quotes, while echo "$(echo '$b')" was? What is the difference between ! and $?
• Why was does echo $(echo '!b') (no quotes) work? (pointed by @MBlanc).

I would prefer to do this without:

• set +H as I would need set -H afterwards to maintain shell state

• backslash escapes, because I need one for every ! and it has to be outside the quotes:

  echo "$(echo '\!a')"
  # '\!a'.
  # No good.
  
  echo "$(echo 'a '\!' b '\!' c')"
  # a ! b ! c
  # Good, but verbose.
  

• echo $(echo '!b') (no quotes), because the command could return spaces.

Version:

bash --version | head -n1
# GNU bash, version 4.2.25(1)-release (i686-pc-linux-gnu)

Answer 1

In your last example,

echo "$(echo '!b')"

the exclamation point is not single-quoted. Because history expansion occurs so early in the parsing process, the single quotes are just part of the double-quoted string; the parser hasn't recognized the command substitution yet to establish a new context where the single quotes would be quoting operators.

To fix, you'll have to temporarily turn off history expansion:

set +H
echo "$(echo '!b')"
set -H

Answer 2

This was repeatedly reported as a bug, most recently against bash 4.3 in 2014, for behavior going back to bash 3.

There was some discussion whether this constituted a bug or expected but perhaps undesirable behavior; it seems the consensus has been that, however you want to characterize the behavior, it shouldn't be allowed to continue.

It's fixed in bash 4.4, echo "$(echo '!b')" doesn't expand, echo "'!b'" does, which I regard as proper behavior because the single quotes are shell syntax markers in the first example and not in the second.

Answer 3

If History Expansion is enabled, you can only echo the ! character if it is put in single quotes, escaped or if followed by a whitespace character, carriage return, or =.

From man bash:

   Only backslash (\) and single quotes can  quote  the  history
   expansion character.

   Several  characters inhibit history expansion if found immediately fol-
   lowing the history expansion character, even if it is unquoted:  space,
   tab,  newline,  carriage return, and =.

I believe the key word here is “Only”. The examples provided in the question only consider the outer most quotes being double quotes.