Is there way to track progress on a mclapply?

Question

I love the setting .progress = \'text\' in plyr\'s llply. However, it causes my much anxiety to not know how far along an mclapp

Answer 1

You can use your systems echo function to write from your workers, so simply add the following line to your function:

myfun <- function(x){
if(x %% 5 == 0) system(paste("echo 'now processing:",x,"'"))
dosomething(mydata[x])
}

result <- mclapply(1:10,myfun,mc.cores=5)
> now processing: 5 
> now processing: 10 

This will work if you pass an index e.g., so rather than passing a list of data, pass the index and extract the data in the worker function.