In bash, when I go back in history, edit some command and run it, this edited command is appended to history and the original one is left intact. But every once in a while I
I somehow manage to affect the original command, i.e. my edit replaces the original command back in history.
Right. If you go back in your history and edit the line without pressing return to execute the command but instead moving to another history entry, you've just edited the history entry. If you then list your history, you will see a * on the line indicating that you edited it. I find this "feature" immensely frustrating. Others have provided good examples of how to reproduce this.
My goal is to avoid this, so any edit to a previous command always gets appended to history and never replaces the original.
I too wanted to disable it. I found the solution via this answer over on unix.stackexchange.
To summarize, you need to enable the revert-all-at-newline readline setting which is off by default. If the setting is on then bash will revert any changes you made to your history when you execute the next command.
To enable this setting in your shell, you should add the following to your ~/.inputrc file and then restart your shell:
$include /etc/inputrc
set revert-all-at-newline on
The first line is needed because I guess that if you supply your own .inputrc file the default /etc/inputrc file is not included which is probably not what you want.