It happens to me all the time. I accidentally version a file, I do not want to be versioned (i.e. developer/machine specific config-files).
If I commit this file, I
Without having tried it...
In git, if your changes haven't been propagated to another repository, you should be able to git rm the affected file(s), git rebase --interactive to reorder the deletion commit to be just after the commit in which you accidentally added the offending files, and then squash those two commits together.
Of course, this won't help if someone else has pulled your changes.