Multithreading a large number of web requests in c#

Question

I have an program where I need to create some large number of folders to an external sharepoint site (external meaning I can\'t use the sharepoint object model). Web request

Answer 1

You're not closing the webrequest which might cause the connection to be open unecessarily long. This sounds like a perfect job for Parallel.Net's Parallel.Foreach, just be sure to indicate how many threads you want it running on

  ParallelOptions parallelOptions = new ParallelOptions();

        parallelOptions.MaxDegreeOfParallelism = 10;
        Parallel.ForEach(folderPathList, parallelOptions, folderPathList =>
        {
            using(WebRequest request = WebRequest.Create(folderPath))
            {
               request.Credentials = DefaultCredentials;
               request.Method = "MKCOL";

               GetResponse request = WebRequest.Create(folderPath);
               request.Credentials = DefaultCredentials;
               request.Method = "MKCOL";
               using (WebResponse response = request.GetResponse());
            }
        });

Another thing to keep in mind is maxConnections, be sure to set it in your app.config:

Of couse in a real-world scenario you would have to add try-catch to and retrying connections that might time out leading to more complicated code