System.Net.WebClient request gets 403 Forbidden but browsers do not with Apache servers

Question

An odd one, I\'m trying to read the section of a lot of different websites out there, and one particular type of server, Apache, sometimes gives the code 403 fo

Answer 1

Try setting the UserAgent header:

string _UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)";
client.Headers.Add(HttpRequestHeader.UserAgent, _UserAgent);