Write output of R loop to file

Question

Currently my R loop (see below) overwrites itself during each iteration. I want to output the result from each loop into a text file.

In more detail: R beginner here

Answer 1

To do it as the loop is running, in your loop add:

write.csv(data.frame(fname=filename[i],mean=mean),file="output.csv",append=TRUE)

However, this would mean a lot of file system overhead, and it would be quicker the produce the whole data frame in R and then write the file as a whole. So instead of your loop write:

means <- sapply(filename, function(x) mean(as.numeric(read.table(x,header=FALSE)$V4)))

And then write the file as a whole with:

write.csv(data.frame(fname=filename,mean=means),file="output.csv")

Answer 2

The following script will do it by first creating the file with only column names, and then appending each result.

filename <- system("ls /dir/",intern=TRUE)

column_names <- data.frame(filename = "filename", mean = "mean")
write.table(column_names, file = "output.csv", row.names = FALSE, 
            append = FALSE, col.names = FALSE, sep = ", ", quote = TRUE)

for(i in 1:length(filename)){
  file <- read.table(filename[i],header=FALSE)
  newline <- data.frame(t(c(filename[i], mean(as.numeric(file$V4)))))
  write.table(newline, file = "output.csv", row.names = FALSE, 
              append = TRUE, col.names = FALSE, sep = ", ")
}

Writing to the file at each step is not very efficient though, and you might consider doing it at the end only:

filename <- system("ls /dir/",intern=TRUE)

results <- data.frame(filename = "filename", mean = "mean")

for(i in 1:length(filename)){
  file <- read.table(filename[i],header=FALSE)
  newline <- data.frame(t(c(filename = filename[i], mean = mean(as.numeric(file$V4)))))
  results <- rbind(results, newline)
}
write.table(results, file = "output.csv", row.names = FALSE, 
            append = FALSE, col.names = TRUE, sep = ", ")