WebClient.DownloadString(url) when this url returns a 404 page, how can i skip this?

Question

I'm using WebClient.DownloadString(url) to download a web page, when a url a 404 web page it stops and doesn't work anymore. I want to skip these pages when I got this fault.

if the url is 404 page, it doesn't start to download. so i can't parse the undownloaded data...

Answer 1

You will have to catch the Exception and test for a 404:

try
{
    string myString;
    using (WebClient wc = new WebClient())
        myString= wc.DownloadString("http://foo.com");

}
catch (WebException ex)
{
    if (ex.Status == WebExceptionStatus.ProtocolError && ex.Response != null)
    {
        var resp = (HttpWebResponse)ex.Response;
        if (resp.StatusCode == HttpStatusCode.NotFound) // HTTP 404
        {
            //the page was not found, continue with next in the for loop
            continue;
        }
    }
    //throw any other exception - this should not occur
    throw;
}

Answer 2

You can put your code in a Try...Catch block and catch a WebException. If you want more control on how to handle specific errors, you can use the WebException's Status property which returns a WebExceptionStatus enumeration.