foreach loop returns only result of first data in the list

问题

I have been struggling about making read of my data from a directory and make a function into a foreach loop,

filelist <- dir(pattern = "*.omf")
readfiles <- function(m){for(k in 1:length(filelist))
file<-filelist[[k]]
data <- read.table("filelist[[k]]", skip=grep('# Begin: Data Text',     readLines("filelist[[k]]")),na.strings=c("NA", "-", "?"),colClasses="numeric")
my <- as.matrix(data[1:57600,2]);
mesh <- array(my, dim = c(120,60,8));
Ms<-1350*10^3    # A/m
asd2=(mesh[70:75,24:36 ,2])/Ms;     # in A/m

ort_my<- mean(asd2);
return(ort_my)
}

# The main function called once each loop

main.fun <- function(m)
{
# Call two other functions
return(readfiles(m))
}  

# Compute the values (odd numbers from 5 to 23) using a foreach loop
compute.cluster <- function(m)
{
values <- foreach(m=1:length(filelist),.combine = "c") %dopar%
{
main.fun(m)
}
return(values)
} 

# Start the cluster and register with doSNOW (node names are just examples)
cl <- makeCluster(12, type = "SOCK")
clusterExport(cl, c("main.fun", "readfiles","filelist"))
registerDoSNOW(cl)

print(compute.cluster())

# Stop the cluster
stopCluster(cl)

When I do this code I get only result of first readlines(m function). Indeed the result is correct for only first of my data. but for others should be different for each k.

> print(compute.cluster())
            [,1]
result.1  -0.2530708
result.2  -0.2530708
result.3  -0.2530708
result.4  -0.2530708
result.5  -0.2530708
result.6  -0.2530708
result.7  -0.2530708
result.8  -0.2530708
result.9  -0.2530708
result.10 -0.2530708
result.11 -0.2530708
result.12 -0.2530708
result.13 -0.2530708
result.14 -0.2530708
result.15 -0.2530708
result.16 -0.2530708
result.17 -0.2530708
result.18 -0.2530708
result.19 -0.2530708
result.20 -0.2530708
result.21 -0.2530708
result.22 -0.2530708
result.23 -0.2530708
result.24 -0.2530708
result.25 -0.2530708
result.26 -0.2530708
result.27 -0.2530708
result.28 -0.2530708
result.29 -0.2530708
result.30 -0.2530708
result.31 -0.2530708
result.32 -0.2530708
result.33 -0.2530708
result.34 -0.2530708
result.35 -0.2530708
result.36 -0.2530708
result.37 -0.2530708
result.38 -0.2530708
result.39 -0.2530708
result.40 -0.2530708
result.41 -0.2530708
result.42 -0.2530708
result.43 -0.2530708
result.44 -0.2530708
result.45 -0.2530708
result.46 -0.2530708
result.47 -0.2530708
result.48 -0.2530708
result.49 -0.2530708
result.50 -0.2530708
result.51 -0.2530708
result.52 -0.2530708
result.53 -0.2530708
result.54 -0.2530708
result.55 -0.2530708
result.56 -0.2530708
result.57 -0.2530708
result.58 -0.2530708
result.59 -0.2530708
result.60 -0.2530708
result.61 -0.2530708
result.62 -0.2530708
result.63 -0.2530708
result.64 -0.2530708
result.65 -0.2530708
result.66 -0.2530708
result.67 -0.2530708
result.68 -0.2530708
result.69 -0.2530708
result.70 -0.2530708

Stop the cluster

stopCluster(cl)

Any help appreciated!

thanks!

回答1:

Have you tested out your functions serially? Do they do what you expected them to do?

I am a little puzzled by the code in your first function (edited):

file <- filelist[[k]]
data <- read.table("filelist[[k]]")

This assigns a file name (string) to the variable file, but the following line attempts to read data from a file named "filelist[[k]]". I am almost 100% certain that this is not what you want... Surely something along these lines would be more appropriate:

file <- filelist[[k]]
data <- read.table(file)

or simply

data <- read.table(filelist[[k]])

Beyond this, my advice would be to get the code working for a serial loop and only then worry about trying to run it in parallel.

Regards, Andrew.

回答2:

I had a look at you're code. But I can't get it to run using your example code above. If you produce a reproducible example, you're more likely to get someone to help you. See: How to make a great R reproducible example?

This example works, made with fake data. But I'm not sure it's what you want.

data.list <- list()
    for (i in 1:10){
      data.list[[i]] <- matrix(data= rnorm(100,1),50,2)
    }

    read.dat <- function(m){
      out <- rep(1:length(data.list))
      for(k in 1:length(data.list)){
      my <- as.matrix(data.list[[k]]);
      ort_my<- mean(my);
      out[k] <- ort_my
    }
      return(out)
    }

# The main function called once each loop
  main.fun <- function(m)
    {
      # Call two other functions
      return(read.dat(m))
    }  

# Start the cluster and register with doSNOW (node names are just examples)
cl <- makeCluster(12, type = "SOCK")
clusterExport(cl, c("main.fun", "read.dat","data.list"))
registerDoSNOW(cl)



 values <- matrix(NA,length(data.list),10)
     foreach(m=1:length(data.list),.combine = "c") %dopar% {
     values[m,] <- main.fun()}

  [1] 1.1507491 1.1226257 0.7971620 1.0954047 0.9954982 0.9721017 0.9738546 0.9596752
  [9] 0.9470697 0.9787103 1.1507491 1.1226257 0.7971620 1.0954047 0.9954982 0.9721017
 [17] 0.9738546 0.9596752 0.9470697 0.9787103 1.1507491 1.1226257 0.7971620 1.0954047
 [25] 0.9954982 0.9721017 0.9738546 0.9596752 0.9470697 0.9787103 1.1507491 1.1226257
 [33] 0.7971620 1.0954047 0.9954982 0.9721017 0.9738546 0.9596752 0.9470697 0.9787103
 [41] 1.1507491 1.1226257 0.7971620 1.0954047 0.9954982 0.9721017 0.9738546 0.9596752
 [49] 0.9470697 0.9787103 1.1507491 1.1226257 0.7971620 1.0954047 0.9954982 0.9721017
 [57] 0.9738546 0.9596752 0.9470697 0.9787103 1.1507491 1.1226257 0.7971620 1.0954047
 [65] 0.9954982 0.9721017 0.9738546 0.9596752 0.9470697 0.9787103 1.1507491 1.1226257
 [73] 0.7971620 1.0954047 0.9954982 0.9721017 0.9738546 0.9596752 0.9470697 0.9787103
 [81] 1.1507491 1.1226257 0.7971620 1.0954047 0.9954982 0.9721017 0.9738546 0.9596752
 [89] 0.9470697 0.9787103 1.1507491 1.1226257 0.7971620 1.0954047 0.9954982 0.9721017
 [97] 0.9738546 0.9596752 0.9470697 0.9787103

 # Stop the cluster
        stopCluster(cl)

来源：https://stackoverflow.com/questions/23682712/foreach-loop-returns-only-result-of-first-data-in-the-list

标签

for-loop

foreach

read.table