doparallel

Consider the very basic (and inefficient) code using parallel foreach for generating random values: cl <- makeCluster(2) registerDoParallel(cl) foreach(i = 1:100) %dopar% rnorm(1) Is it correct or are there any additional steps needed for random generation to work properly? I guess it's enough and fast checks seem to "prove" that seeds work properly, but I'd like to be sure that it is so on other platforms, since I want the code to be portable. Your worries are correct; random number generation does not magically work in parallel and further steps need to be taken. When using the foreach

I would like to use the "foreach" loop in R (package foreach + doParallel) but in my work i found that the loop returns some NA and the classic "for" loop returns the value I want : library(foreach) library(doParallel) ncore=as.numeric(Sys.getenv('NUMBER_OF_PROCESSORS'))-1 registerDoParallel(cores=ncore) B=2 a = vector() b = vector() foreach(i = 1:B, .packages = "ez",.multicombine = T,.inorder = T, .combine = 'c')%dopar%{ a[i] = i + 1 return(a) } for(i in 1:B){ b[i] = i + 1 b } As you can see if you try it, the object "a" returns a vector with 2, NA and 3 while the object "b" returns 2 and 3

Is there a way to modify how R foreach loop does load balancing with doParallel backend ? When parallelizing tasks that have very different execution time, it can happen all nodes but one have finished their tasks while the last one still have several tasks to do. Here is a toy example: library(foreach) library(doParallel) registerDoParallel(4) waittime = c(10,1,1,1,10,1,1,1,10,1,1,1,10,1,1,1) w = iter(waittime) foreach(i=w) %dopar% { message(paste("waiting",i, "on",Sys.getpid())) Sys.sleep(i) } Basically, the code register 4 cores. For each loop i , the task is to wait for waittime[i] seconds

I ran into an issue trying to use %dopar% and foreach() together with an R6 class. Searching around, I could only find two resources related to this, an unanswered SO question and an open GitHub issue on the R6 repository. In one comment (i.e., GitHub issue) an workaround is suggested by reassigning the parent_env of the class as SomeClass$parent_env <- environment() . I would like to understand what exactly does environment() refer to when this expression (i.e., SomeClass$parent_env <- environment() ) is called within the %dopar% of foreach ? Here is a minimal reproducible example: Work <- R6

I am currently working with some large datasets, so parallelizing the workflows is the only way to go. I need to load some packages to each thread once at the beginning (i.e: for(this.thread in threads) { #load some packages } . Unfortunately , I'm not sure how to do that. The following code further illustrates my problem, where I am trying to use the pipe operator from magrittr in a %dopar% : . library(parallel) library(doParallel) library(foreach) library(magrittr) # Generate some random data and function : # ----------------------------------------- randomData = runif(10^3) randomFunction =

I need to perform some operations on a tensor and I would like make this parallel. Consider the following example: # first part without doParallel N = 8192 M = 128 F = 64 ma <- function(x,n=5){filter(x,rep(1/n,n), sides=2)} m <- array(rexp(N*M*F), dim=c(N,M,F)) new_m <- array(0, dim=c(N,M,F)) system.time ( for(i in 1:N) { for(j in 1:F) { ma_r <- ma(m[i,,j],2) ma_r <- c(ma_r[-length(ma_r)], ma_r[(length(ma_r)-1)]) new_m[i,,j] <- ma_r } } ) This takes around 38 seconds in my laptop. The following is with doParallel: # second part with doParallel library(doParallel) no_cores <- detectCores() - 1

I'm new to using foreach() %dopar% for paralleling, and I have some problems about how it handles errors or warnings. when I use try() with my customized error message within foreach() %dopar%, the "native" error message doesn't show up: test <- function(x) { if (x==2) "a"/2 } foreach(i=1:3) %dopar% { tryout <- try(test(i)) if (class(tryout)=="try-error") print("Error!") } In this case the "native" error message: Error in "a"/2 : non-numeric argument to binary operator doesn't show up, and only the Error! from try() error catching will be printed. However both error messages will be printed