Can rbind be parallelized in R?

Question

As I am sitting here waiting for some R scripts to run...I was wondering... is there any way to parallelize rbind in R?

I sitting waiting for this call to complete

Answer 1

Because you said that you want to rbind data.frame objects you should use the data.table package. It has a function called rbindlist that enhance drastically rbind. I am not 100% sure but I would bet any use of rbind would trigger a copy when rbindlist does not. Anyway a data.table is a data.frame so you do not loose anything to try.

EDIT:

library(data.table)
system.time(dt <- rbindlist(pieces))
utilisateur     système      écoulé 
       0.12        0.00        0.13 
tables()
     NAME  NROW MB COLS                        KEY
[1,] dt   1,000 8  X1,X2,X3,X4,X5,X6,X7,X8,...    
Total: 8MB

Lightning fast...