parallel-foreach

Issue on using R6 classes and foreach() %dopar% together

由 我与影子孤独终老i 提交于 2019-12-23 22:52:50
问题 I'm having an issue on R6 classes when used with foreach() together, possibly to do with environments (I'm using Windows). Suppose that there are two R6 classes, "class1" and "class2". method1 in class1 is dependent on class2 (see example code below for example). The issue is, if I use foreach() %dopar% on class1, R doesn’t seem to recognise class2, even if I set .export = c("class1", "class2") explicitly in foreach() statement. (Here class1 uses class2) However if I use foreach() on class2,

run a for loop in parallel in R

由 南笙酒味 提交于 2019-12-17 07:07:06
问题 I have a for loop that is something like this: for (i=1:150000) { tempMatrix = {} tempMatrix = functionThatDoesSomething() #calling a function finalMatrix = cbind(finalMatrix, tempMatrix) } Could you tell me how to make this parallel ? I tried this based on an example online, but am not sure if the syntax is correct. It also didn't increase the speed much. finalMatrix = foreach(i=1:150000, .combine=cbind) %dopar% { tempMatrix = {} tempMatrix = functionThatDoesSomething() #calling a function

Error in { : task 3 failed - “RcallMethod: attempt to call a method of a NULL object.”

由 扶醉桌前 提交于 2019-12-13 03:57:23
问题 Any help of this would be greatly appreciated. I have the following R code (following tutorial at https://www.r-bloggers.com/parallel-r-loops-for-windows-and-linux/) to carry out 233 mutually exclusive calculations in parallel. There are 4 cores and 32 GB of RAM on the machine where this code is executing. # Calculate the number of cores no_cores <- detectCores() - 1 # initiate cluster cl <- makeCluster(no_cores) registerDoSNOW(cl) # Process sandboxes in parallel - there are 233 sandboxes in

Specify package location in foreach

由 北慕城南 提交于 2019-12-11 09:39:27
问题 I am installing a package to a particular directory and then loading in the library using: library(CustomPackage, lib.loc = "R_Libs") Then when using foreach I am having trouble figuring out how to load this one package from that custom location "R_Libs". foreach(i=(1:100), .packages=c("lubridate","CustomPackage")) %dopar% { some code here... } Any ideas how to force that one package to be read from the "R_Libs" directory? 回答1: Modifying library paths in R console is meaningless. > library

Parallel Processing, inside loop, can we access the Degree of Parallelism?

由 删除回忆录丶 提交于 2019-12-11 04:24:40
问题 In the following loop, I want to add logic to say if processor 1 then do this, if processor 2 do that etc. I have been trying various properties like ... Console.WriteLine("Domain ID = " + Thread.GetDomainID().ToString()); Console.WriteLine("Thread ID = " + Thread.CurrentThread.ManagedThreadId.ToString()); ...but i see thread ID is counting up and up so its the thread where many threads to a processor. So how to get the 1-4 inside the loop? The DomainID I just get the same value Parallel

foreach, doParallel and random generation

由 此生再无相见时 提交于 2019-12-08 19:07:27
Consider the very basic (and inefficient) code using parallel foreach for generating random values: cl <- makeCluster(2) registerDoParallel(cl) foreach(i = 1:100) %dopar% rnorm(1) Is it correct or are there any additional steps needed for random generation to work properly? I guess it's enough and fast checks seem to "prove" that seeds work properly, but I'd like to be sure that it is so on other platforms, since I want the code to be portable. Your worries are correct; random number generation does not magically work in parallel and further steps need to be taken. When using the foreach

Nested for loops in R using foreach function and doParallel library

由 纵然是瞬间 提交于 2019-12-08 07:23:01
问题 I am trying to calculate the cosine similarity between columns in a matrix. I am able to get it to work using standard for loops, but when I try to make it run in parallel to make the code run faster it does not give me the same answer. The problem is that I am unable to get the same answer using the foreach loop approach. I suspect that I am not using the correct syntax, because I have had single foreach loops work. I have tried to make the second loop a regular for loop and I used the %:%

foreach, doParallel and random generation

由 你说的曾经没有我的故事 提交于 2019-12-08 03:57:09
问题 Consider the very basic (and inefficient) code using parallel foreach for generating random values: cl <- makeCluster(2) registerDoParallel(cl) foreach(i = 1:100) %dopar% rnorm(1) Is it correct or are there any additional steps needed for random generation to work properly? I guess it's enough and fast checks seem to "prove" that seeds work properly, but I'd like to be sure that it is so on other platforms, since I want the code to be portable. 回答1: Your worries are correct; random number

Output list of two rbinded data frames with foreach in R

由 送分小仙女□ 提交于 2019-12-07 11:08:54
问题 Let's say I want to use foreach in the doParallel package to return a list of two data frames of different dimensions like the following: a<-NULL b<-NULL for(i in 1:100){ a<-rbind(a,data.frame(input=i,output=i/2)) if(i > 5){ b<-rbind(b,data.frame(input=i,output=i^2)) } } list(a,b) Since foreach returns an object, there's no (at least to me) obvious way to do the above with foreach . NOTE: this is a much simplified version of the problem I'm actually working with so solving the problem by

Why is R for loop 10 times slower than when using foreach?

由 青春壹個敷衍的年華 提交于 2019-12-06 19:46:10
问题 This is really blowing my mind. The basic loop takes like 8 seconds on my computer: system.time({ x <- 0 for (p in 1:2) { for (i in 1:500) { for (j in 1:5000) { x <- x + i * j } } } }) x Whereas if I use foreach in non-parallel mode, it does take only 0.7 secs!!! system.time({ x <- 0 foreach(p = 1:2, .combine = rbind) %do% for (i in 1:500) { for (j in 1:5000) { x <- x + i * j } } }) x The result is the same, but foreach was somehow able to reach it much faster than basic R! Where is the
订阅 parallel-foreach