R - problem with foreach %dopar% inside function called by optim

Question

Calling a function that includes foreach %dopar% construct from optim causes an error:

> workers <- startWorkers(6) # 6 cores
> 
> registerDoSMP(         


        

Answer 1

I ran into the same problem an the issue is with the environment not being included in the sub-threads. Your error

Error in { : task 1 failed - "could not find function "simple_fn""

can be reproduced by this very simple example:

simple_fn <- function(x)
    x+1

test_par <- function(){
    library("parallel")
    no_cores <- detectCores()
    library("foreach")
    cl<-makeCluster(no_cores)
    library("doSNOW")
    registerDoSNOW(cl)
    out <- foreach(i=1:10) %dopar% {
        simple_fn(i)
    }

    stopCluster(cl)
    return(out)
}

test_par()

Now all you need to to is to change the foreach(i=1:10) into foreach(i=1:10, .export=c("simple_fn")). If you want to export your complete global environment then just write .export=ls(envir=globalenv()) and you will have it for better or worse.