The man page for Bash says, regarding the -c option:
-c stringIf the-coption is present, then comma
Because '$0' and '$1' in your string is replaced with a variable #0 and #1 respectively.
Try :
bash -c "echo arg 0: \$0, arg 1: \$1" arg0 arg1
In this code $ of both are escape so base see it as a string $ and not get replaced.
The result of this command is:
arg 0: arg0, arg 1: arg1
Hope this helps.