WebClient 403 Forbidden

Question

I can download this by hand in IE.

http://scholar.google.com/scholar.ris?q=info:j8ymU9rzMsEJ:scholar.google.com/&output=citation&hl=zh-CN&as_sdt=2000&

Answer 1

I had this problem trying to download an image from a SharePoint site url. In my case setting the user-agent to Other or blank in the header wasn't enough, I had to set the user-agent as follows instead:

client.Headers.Add("user-agent", " Mozilla/5.0 (Windows NT 6.1; WOW64; rv:25.0) Gecko/20100101 Firefox/25.0");

That solution came from this answer.

Answer 2

I ran into the same issue trying to download a file on an Amazon 3S url. I blogged about it here: http://blog.cdeutsch.com/2010/11/net-webclient-403-forbidden-error.html

The final solution I used was found here: GETting a URL with an url-encoded slash

Answer 3

The key to solving this for me was to do the request once via code, a second time in the browser, log both requests with Fiddler and ensure the headers match up.

I ended up having to add headers for:

• Accept
• Accept-Encoding
• Accept-Language
• User-Agent
• Upgrade-Insecure-Requests

I hope this helps people in the future.

Answer 4

403 may also be caused by TLS issues. To verify, you should check the text of the WebException.Response object.

     catch (WebException ex)
     {
        if (ex.Response != null)
        {
           var response = ex.Response;
           var dataStream = response.GetResponseStream();
           var reader = new StreamReader(dataStream);
           var details = reader.ReadToEnd();
        }
     }

If it is TLS then try adding this to your code to force TLS1.2.

For .net4:

ServicePointManager.SecurityProtocol = (SecurityProtocolType)3072;

For .net4.5 or later:

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;

Answer 5

You need to set appropriate http headers before calling your DownloadFile method.

WebClient webClient = new WebClient();
webClient.Headers.Add("???", "???");
webClient.Headers.Add("???", "???");
webClient.Headers.Add("???", "???");
webClient.DownloadFile(address, filename);

To put correct values instead of these question marks might be tricky. You will need to download Fiddler or some other program or webbrowser extension to reveal what http headers are being sent to Google by your webbrowser and basically replicate the same request in your program.

Answer 6

This is what happened with me:

I was trying to download a (public) .xls file (via DownloadFile method) which was getting downloaded comfortably from all browsers.

After trying and struggling with all answers (but no luck), I finally opened the Stack and noticed something odd (refer screenshot).

Although, the file was getting downloaded via http in browser but it was giving 403 error via DownloadFile method.

Finally, I just changed the URL from http://something to https://something and it worked fine.

Hope this helps!