Normally, one writes code something like this to download some data using a WebRequest.
using(WebResponse resp = request.GetResponse()) // WebRequest reques
A very interesting question (although worth pointing out that the WebResponse object will have been disposed as you exit the using). My gut feeling is that it doesn't really matter that you've got a reference to this disposed WebResponse object as long as you don't try to do anything "operational" with it.
You can probably still access certain properties on the instance for logging purposes (such as the ResponseUri) without getting an ObjectDisposedException, but overall reference held by the exception is not there so you can continue using the instance.
I'd be interested to see what others say.