Let\'s say, you have a Bash alias like:
alias rxvt=\'urxvt\'
which works fine.
However:
I can confirm that using '\'' for a single quote inside a single-quoted string does work in Bash, and it can be explained in the same way as the "gluing" argument from earlier in the thread. Suppose we have a quoted string: 'A '\''B'\'' C' (all quotes here are single quotes). If it is passed to echo, it prints the following: A 'B' C.
In each '\'' the first quote closes the current single-quoted string, the following \' glues a single quote to the previous string (\' is a way to specify a single quote without starting a quoted string), and the last quote opens another single-quoted string.