I have created a sample asp.net application and try to scrape data from my server using httpwebrequest. But some times I got this above error. I have done some searches on g
You may want to check the content of your headers. Try adding the following, as suggested by this link:
Accept: text/html, application/xhtml+xml, */*