parallel-foreach

I am trying to calculate the cosine similarity between columns in a matrix. I am able to get it to work using standard for loops, but when I try to make it run in parallel to make the code run faster it does not give me the same answer. The problem is that I am unable to get the same answer using the foreach loop approach. I suspect that I am not using the correct syntax, because I have had single foreach loops work. I have tried to make the second loop a regular for loop and I used the %:% parameter with the foreach loop, but then the function doesn't even run. Please see my attached code

I ran into an issue trying to use %dopar% and foreach() together with an R6 class. Searching around, I could only find two resources related to this, an unanswered SO question and an open GitHub issue on the R6 repository. In one comment (i.e., GitHub issue) an workaround is suggested by reassigning the parent_env of the class as SomeClass$parent_env <- environment() . I would like to understand what exactly does environment() refer to when this expression (i.e., SomeClass$parent_env <- environment() ) is called within the %dopar% of foreach ? Here is a minimal reproducible example: Work <- R6

This is really blowing my mind. The basic loop takes like 8 seconds on my computer: system.time({ x <- 0 for (p in 1:2) { for (i in 1:500) { for (j in 1:5000) { x <- x + i * j } } } }) x Whereas if I use foreach in non-parallel mode, it does take only 0.7 secs!!! system.time({ x <- 0 foreach(p = 1:2, .combine = rbind) %do% for (i in 1:500) { for (j in 1:5000) { x <- x + i * j } } }) x The result is the same, but foreach was somehow able to reach it much faster than basic R! Where is the inefficiency of basic R? How is this possible? In fact, I got complete opposite result compared to this one:

Based on this post here , I tried to write a script, seen here: library(parallel) library(doParallel) cl<-makeCluster(2,outfile='') registerDoParallel(cl) foreach(i=1:5, .packages='parallel') %dopar% { system.time(mclapply(1:10, function(x){rnorm(1e5)},mc.cores=2)) } stopCluster(cl) It worked intially but is now throwing up error codes: Error in unserialize(node$con) : error reading from connection Calls: <Anonymous> ... doTryCatch -> recvData -> recvData.SOCKnode -> unserialize Execution halted Error in unserialize(socklist[[n]]) : error reading from connection Error in unserialize(node$con)

I'm running the following code (extracted from doParallel's Vignettes ) on a PC (OS Linux) with 4 and 8 physical and logical cores, respectively. Running the code with iter=1e+6 or less, every thing is fine and I can see from CPU usage that all cores are employed for this computation. However, with larger number of iterations (e.g. iter=4e+6 ), it seems parallel computing does not work in which case. When I also monitor the CPU usage, just one core is involved in computations (100% usage). Example1 require("doParallel") require("foreach") registerDoParallel(cores=8) x <- iris[which(iris[,5] !=

I am trying to read all lines in a text file and planning to display each line info. How can I find the index for each item inside loop? string[] lines = File.ReadAllLines("MyFile.txt"); List<string> list_lines = new List<string>(lines); Parallel.ForEach(list_lines, (line, index) => { Console.WriteLine(index); // Console.WriteLine(list_lines[index]); Console.WriteLine(list_lines[0]); }); Console.ReadLine(); Curtis Lusmore There is another overload for Parallel.ForEach that gives you the index. Parallel.ForEach(list_lines, (line, state, index) => { Console.WriteLine(index); Console.WriteLine

I have a for loop that is something like this: for (i=1:150000) { tempMatrix = {} tempMatrix = functionThatDoesSomething() #calling a function finalMatrix = cbind(finalMatrix, tempMatrix) } Could you tell me how to make this parallel ? I tried this based on an example online, but am not sure if the syntax is correct. It also didn't increase the speed much. finalMatrix = foreach(i=1:150000, .combine=cbind) %dopar% { tempMatrix = {} tempMatrix = functionThatDoesSomething() #calling a function cbind(finalMatrix, tempMatrix) } Thanks for your feedback. I did look up parallel after I posted this