“Unable to connect to remote server fail” in HttpWebRequest

Question

I am using VSTS 2008 + C# + .Net 3.5 to develop a console application and I send request to another server (IIS 7.0 on Windows Server 2008). I find when the # of request thr

Answer 1

Kev answered you question already, I just want to add that creating so many threads is not really good design solution (just context switching overhead is a big minus) plus it won't scale good.

The quick answer would be: use asynchronous operations to read data instead of creating a bunch of threads. Or at least use thread pool (and lower worker thread count). Remember that more connections to one source will only speed things up till some degree. Try benchmarking it and you will see that probably 3-5 connections will work faster that 2000 you are using now.

You can read more about asynchronous client/server architecture (IOCP - input/output completion ports) and its advantages here. You can start from here:

MSDN - Using an Asynchronous Server Socket

MSDN - Asynchronous Server Socket Example

CodeProject - Multi-threaded .NET TCP Server Examples

All of these examples uses lower level TCP object, but it can be applied to WebRequest/WebResponse as well.

UPDATE

To try thread pool version, you can do something like this:

ManualResetEvent[] events = new ManualResetEvent[ClientCount];
for (uint cnt  = 0; cnt < events.Length; cnt++)
{
  events[cnt] = new ManualResetEvent(false);
  ThreadPool.QueueUserWorkItem(obj => PerformanceWorker());
}

WaitHandle.WaitAll(events);

Not tested, may need some adjustment.