EDIT: the command substitution is not necessary for the surprising behavior, although it is the most common use case. The same question applies to just echo \"\'!b\'\"
In your last example,
echo "$(echo '!b')"
the exclamation point is not single-quoted. Because history expansion occurs so early in the parsing process, the single quotes are just part of the double-quoted string; the parser hasn't recognized the command substitution yet to establish a new context where the single quotes would be quoting operators.
To fix, you'll have to temporarily turn off history expansion:
set +H
echo "$(echo '!b')"
set -H