Parallel.Foreach spawning way too many threads

Question

The problem

Although the code about which I will talk here I wrote in F#, it is based on the .NET 4 framework, not specifically depending on any particularity of F

Answer 1

Using 'async's will enable you to do the I/O-bound work without burning threads while the various I/O calls are 'at sea', so that would be my first suggestion. It should be straightforward to convert the code to async, usually along the lines of

• wrap each function body in async{...}, add return where necessary
• create Async versions of any I/O primitives that aren't already in the library via Async.FromBeginEnd
• Switch calls of the form let r = Foo() to let! r = AsyncFoo()
• Use Async.Parallel to convert the 5000 async objects into a single Async that runs in parallel

There are various tutorials for doing this; one such webcast is here.