It happens to me all the time. I accidentally version a file, I do not want to be versioned (i.e. developer/machine specific config-files).
If I commit this file, I
In Git, in order to delete it from the tree, but NOT from the working directory, which I think is what you want, you can use the --cached flag, that is:
git rm --cached