Parallel execution for IO bound operations

Question

I have read TPL and Task library documents cover to cover. But, I still couldn\'t comprehend the following case very clearly and right now I need to implement it.

I

Answer 1

In this case can I still use Parallel.Foreach ?

This isn't really appropriate. Parallel.Foreach is more for CPU intensive work. It also doesn't support async operations.

In case of using Task instead, what is the best practice for creating huge number of them?

Use a TPL Dataflow block instead. You don't create huge amounts of tasks that sit there waiting for a thread to become available. You can configure the max amount of tasks and reuse them for all the items that meanwhile sit in a buffer waiting for a task. For example:

var block = new ActionBlock(
    uri => SendRequestAsync(uri),
    new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 50 });

foreach (var uri in uris)
{
    block.Post(uri);
}

block.Complete();
await block.Completion;