I\'m getting a strange error on my webserver for seemingly every file but the .aspx files.
Here is an example. Just replace \'/robots.txt\' with any .jpg name or .gi
You can ignore robots.txt and all the aspx pages in your routing.
routes.IgnoreRoute("{*allaspx}", new {allaspx=@".*\.aspx(/.*)?"});
routes.IgnoreRoute("{*robotstxt}", new {robotstxt=@"(.*/)?robots.txt(/.*)?"});
You might want to ignore the favicon too.
routes.IgnoreRoute("{*favicon}", new {favicon=@"(.*/)?favicon.ico(/.*)?"});
You can adjust the regular expression to exclude paths.
Haacked from the source.