System.Net.WebClient unreasonably slow

Question

When using the System.Net.WebClient.DownloadData() method I\'m getting an unreasonably slow response time.

When fetching an url using the WebClient class in .NET it

Answer 1

There is nothing inherently slow about .NET web requests; that code should be fine. I regularly use WebClient and it works very quickly.

How big is the payload in each direction? Silly question maybe, but is it simply bandwidth limitations?

IMO the most likely thing is that your web-site has spun down, and when you hit the URL the web-site is slow to respond. This is then not the fault of the client. It is also possible that DNS is slow for some reason (in which case you could hard-code the IP into your "hosts" file), or that some proxy server in the middle is slow.

If the web-site isn't yours, it is also possible that they are detecting atypical usage and deliberately injecting a delay to annoy scrapers.

I would grab Fiddler (a free, simple web inspector) and look at the timings.