Why does iterating over GetConsumingEnumerable() not fully empty the underlying blocking collection

Question

I have a quantifiable & repeatable problem using the Task Parallel Library, BlockingCollection, ConcurrentQueue & GetCo

Answer 1

I couldn't replicate your behavior with simple console application doing basically the same thing (running on .Net 4.5 beta, which could make a difference). But I think the reason this happens is that Parallel.ForEach() tries to optimize execution by splitting the input collection into chunks. And with your enumerable, a chunk can't be created until you add more items to the collection. For more information, see Custom Partitioners for PLINQ and TPL on MSDN.

To fix this, don't use Parallel.ForEach(). If you still want to process the items in parallel, you can start a Task in each iteration.