htmlagilitypack gzip encryption exception

Question

I'm having the exception throw gzip is not support. This is all i'm using the load the page, any idea on how to allow gzip?

        HtmlWeb hwObject = new HtmlWeb();
        HtmlAgilityPack.HtmlDocument htmldocObject = hwObject.Load(siteURL);

Answer 1

You can download the page yourself, i.e. using a class derived from WebClient (or manually making a WebRequest and setting AutomaticDecompression )

public class GZipWebClient : WebClient
{
    protected override WebRequest GetWebRequest(Uri address)
    {
        HttpWebRequest request = (HttpWebRequest)base.GetWebRequest(address);
        request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
        return request;
    }
}

Given this you can do:

string html;
using(var wc = new GZipWebClient())
  html = wc.DownloadString(siteUrl);

var htmldocObject = new HtmlDocument();
htmldocObject.LoadHtml(html);