I have a VS 2005 web site that I publish using \"Publish Web Site\", and I clear all the three checkboxes. I also have a deployment project that picks up the published files
This option solved the problem for me. Basically, it removes every orphan file left after previous deploy.