Mean of elements in a list of data.frames

Question

Suppose I had a list of data.frames (of equal rows and columns)

dat1 <- as.data.frame(matrix(rnorm(25), ncol=5))
dat2 <- as.data.frame(matrix(rnorm(25), ncol=5))
dat3 <- as.data.frame(matrix(rnorm(25), ncol=5))

all.dat <- list(dat1=dat1, dat2=dat2, dat3=dat3)

How can I return a single data.frame that is the mean (or sum, etc.) for each element in the data.frames across the list (e.g., mean of first row and first column from lists 1, 2, 3 and so on)? I have tried lapply and ldply in plyr but these return the statistic for each data.frame within the list.

Edit: For some reason, this was retagged as homework. Not that it matters either way, but this is not a homework question. I just don't know why I can't get this to work. Thanks for any insight!

Edit2: For further clarification: I can get the results using loops, but I was hoping that there were a way (a simpler and faster way because the data I am using has data.frames that are 12 rows by 100 columns and there is a list of 1000+ of these data frames).

z <- matrix(0, nrow(all.dat$dat1), ncol(all.dat$dat1))

for(l in 1:nrow(all.dat$dat1)){
   for(m in 1:ncol(all.dat$dat1)){
      z[l, m] <- mean(unlist(lapply(all.dat, `[`, i =l, j = m)))
   }
}

With a result of the means:

> z
        [,1]        [,2]        [,3]        [,4]       [,5]
[1,] -0.64185488  0.06220447 -0.02153806  0.83567173  0.3978507
[2,] -0.27953054 -0.19567085  0.45718399 -0.02823715  0.4932950
[3,]  0.40506666  0.95157856  1.00017954  0.57434125 -0.5969884
[4,]  0.71972821 -0.29190645  0.16257478 -0.08897047  0.9703909
[5,] -0.05570302  0.62045662  0.93427522 -0.55295824  0.7064439

I was wondering if there was a less clunky and faster way to do this. Thanks!

Answer 1

Here is a one liner with plyr. You can replace mean with any other function that you want.

ans1 = aaply(laply(all.dat, as.matrix), c(2, 3), mean)

Answer 2

You would have an easier time changing the data structure, combining the three two dimensional matrices into a single 3 dimensional array (using the abind library). Then the solution is more direct using apply and specifying the dimensions to average over.

EDIT:

When I answered the question, it was tagged homework, so I just gave an approach. The original poster removed that tag, so I will take him/her at his/her word that it isn't.

library("abind")

all.matrix <- abind(all.dat, along=3)
apply(all.matrix, c(1,2), mean)

Answer 3

I gave one answer that uses a completely different data structure to achieve the result. This answer uses the data structure (list of data frames) given directly. I think it is less elegant, but wanted to provide it anyway.

Reduce(`+`, all.dat) / length(all.dat)

The logic is to add the data frames together element by element (which + will do with data frames), then divide by the number of data frames. Using Reduce is necessary since + can only take two arguments at a time (and addition is associative).

Answer 4

Another approach using only base functions to change the structure of the object:

listVec <- lapply(all.dat, c, recursive=TRUE)
m <- do.call(cbind, listVec)

Now you can calculate the mean with rowMeans or the median with apply:

means <- rowMeans(m)
medians <- apply(m, 1, median)

Answer 5

I would take a slightly different approach:

library(plyr)
tmp <- ldply(all.dat) # convert to df
tmp$counter <- 1:5 # 1:12 for your actual situation
ddply(tmp, .(counter), function(x) colMeans(x[2:ncol(x)]))

Answer 6

Couldn't you just use nested lapply() calls?

This appears to give the correct result on my machine

mean.dat <- lapply(all.dat, function (x) lapply(x, mean, na.rm=TRUE))