parallel.foreach

Garbage Collection and Parallel.ForEach Issue After VS2015 Upgrade

柔情痞子 提交于 2019-11-27 13:27:53
I have some code to process several million data rows in my own R-like C# DataFrame class. There's a number of Parallel.ForEach calls for iterating over the data rows in parallel. This code has been running for over a year using VS2013 and .NET 4.5 without issues. I have two dev machines (A and B) and recently upgraded machine A to VS2015. I started noticing a strange intermittent freeze in my code about half the time. Letting it run for a long time, it turns out that the code does eventually finish. It just takes 15-120 minutes instead of 1-2 minutes. Attempts to Break All using the VS2015

Break parallel.foreach?

。_饼干妹妹 提交于 2019-11-27 11:37:34
How do I break out of an parallel.for loop? I have a pretty complex statement which looks like the following: Parallel.ForEach<ColorIndexHolder>(ColorIndex.AsEnumerable(), new Action<ColorIndexHolder>((ColorIndexHolder Element) => { if (Element.StartIndex <= I && Element.StartIndex + Element.Length >= I) { Found = true; break; } })); Using parallel class, I can optimize this process by far. However; I cannot figure out how to break the parallel loop? The break; statement throws following syntax error: No enclosing loops out of which to break or continue Use the ParallelLoopState.Break method:

Parallel foreach with asynchronous lambda

早过忘川 提交于 2019-11-27 10:22:11
I would like to handle a collection in parallel, but I'm having trouble implementing it and I'm therefore hoping for some help. The trouble arises if I want to call a method marked async in C#, within the lambda of the parallel loop. For example: var bag = new ConcurrentBag<object>(); Parallel.ForEach(myCollection, async item => { // some pre stuff var response = await GetData(item); bag.Add(response); // some post stuff } var count = bag.Count; The problem occurs with the count being 0, because all the threads created are effectively just background threads and the Parallel.ForEach call doesn

Parallel.ForEach() vs. foreach(IEnumerable<T>.AsParallel())

老子叫甜甜 提交于 2019-11-27 05:02:00
问题 Erg, I'm trying to find these two methods in the BCL using Reflector, but can't locate them. What's the difference between these two snippets? A: IEnumerable<string> items = ... Parallel.ForEach(items, item => { ... }); B: IEnumerable<string> items = ... foreach (var item in items.AsParallel()) { ... } Are there different consequences of using one over the other? (Assume that whatever I'm doing in the bracketed bodies of both examples is thread safe.) 回答1: They do something quite different.

What does MaxDegreeOfParallelism do?

﹥>﹥吖頭↗ 提交于 2019-11-26 20:51:04
I am using Parallel.ForEach and I am doing some database updates, now without setting MaxDegreeOfParallelism , a dual core processor machine results in sql client timeouts, where else quad core processor machine somehow does not timeout. Now I have no control over what kind of processor cores are available where my code runs, but is there some settings I can change with MaxDegreeOfParallelism that will probably run less operations simultaneously and not result in timeouts? I can increase timeouts but it isnt a good solution, if on lower CPU I can process less operations simultaneously, that

Parallel.ForEach Slower than ForEach

こ雲淡風輕ζ 提交于 2019-11-26 18:46:00
Here is the code: using (var context = new AventureWorksDataContext()) { IEnumerable<Customer> _customerQuery = from c in context.Customers where c.FirstName.StartsWith("A") select c; var watch = new Stopwatch(); watch.Start(); var result = Parallel.ForEach(_customerQuery, c => Console.WriteLine(c.FirstName)); watch.Stop(); Debug.WriteLine(watch.ElapsedMilliseconds); watch = new Stopwatch(); watch.Start(); foreach (var customer in _customerQuery) { Console.WriteLine(customer.FirstName); } watch.Stop(); Debug.WriteLine(watch.ElapsedMilliseconds); } The problem is, Parallel.ForEach takes about

C# Download data from huge list of urls [duplicate]

你。 提交于 2019-11-26 16:48:00
问题 This question already has answers here : Mass Downloading of Webpages C# (7 answers) Closed 6 years ago . I have a huge list of web pages which display a status, which i need to check. Some urls are within the same site, another set is located on another site. Right now i'm trying to do this in a parallel way by using code like below, but i have the feeling that i'm causing too much overhead. while(ListOfUrls.Count > 0){ Parallel.ForEach(ListOfUrls, url => { WebClient webClient = new

Parallel.ForEach and async-await

ⅰ亾dé卋堺 提交于 2019-11-26 16:37:20
I had such method: public async Task<MyResult> GetResult() { MyResult result = new MyResult(); foreach(var method in Methods) { string json = await Process(method); result.Prop1 = PopulateProp1(json); result.Prop2 = PopulateProp2(json); } return result; } Then I decided to use Parallel.ForEach : public async Task<MyResult> GetResult() { MyResult result = new MyResult(); Parallel.ForEach(Methods, async method => { string json = await Process(method); result.Prop1 = PopulateProp1(json); result.Prop2 = PopulateProp2(json); }); return result; } But now I've got an error: An asynchronous module or

Break parallel.foreach?

左心房为你撑大大i 提交于 2019-11-26 15:36:31
问题 How do I break out of an parallel.for loop? I have a pretty complex statement which looks like the following: Parallel.ForEach<ColorIndexHolder>(ColorIndex.AsEnumerable(), new Action<ColorIndexHolder>((ColorIndexHolder Element) => { if (Element.StartIndex <= I && Element.StartIndex + Element.Length >= I) { Found = true; break; } })); Using parallel class, I can optimize this process by far. However; I cannot figure out how to break the parallel loop? The break; statement throws following

Parallel foreach with asynchronous lambda

╄→尐↘猪︶ㄣ 提交于 2019-11-26 15:06:20
问题 I would like to handle a collection in parallel, but I'm having trouble implementing it and I'm therefore hoping for some help. The trouble arises if I want to call a method marked async in C#, within the lambda of the parallel loop. For example: var bag = new ConcurrentBag<object>(); Parallel.ForEach(myCollection, async item => { // some pre stuff var response = await GetData(item); bag.Add(response); // some post stuff } var count = bag.Count; The problem occurs with the count being 0,