Here is a snippet of the code :
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(request.RawUrl);
WebRequest.DefaultWebProxy = null;//Ensure that
There are some good solutions here, but they all seem to be trying to parse the charset out of the content type string. Here's a solution using System.Net.Mime.ContentType, which should be more reliable, and shorter.
var client = new System.Net.WebClient();
var data = client.DownloadData(url);
var encoding = System.Text.Encoding.Default;
var contentType = new System.Net.Mime.ContentType(client.ResponseHeaders[HttpResponseHeader.ContentType]);
if (!String.IsNullOrEmpty(contentType.CharSet))
{
encoding = System.Text.Encoding.GetEncoding(contentType.CharSet);
}
string result = encoding.GetString(data);