Speed up RData load

后端 未结 3 1216
清酒与你
清酒与你 2020-12-23 21:15

I\'ve checked several related questions such is this

How to load data quickly into R?

I\'m quoting specific part of the most rated answer

3条回答
  •  醉酒成梦
    2020-12-23 21:41

    For variables that big, I suspect that most of the time is taken up inside the internal C code (http://svn.r-project.org/R/trunk/src/main/saveload.c). You can run some profiling to see if I'm right. (All the R code in the load function does is check that your file is non-empty and hasn't been corrupted.

    As well as reading the variables into memory, they (amongst other things) need to be stored inside an R environment.

    The only obvious way of getting a big speedup in loading variables would be to rewrite the code in a parallel way to allow simultaneous loading of variables. This presumably requires a substantial rewrite of R's internals, so don't hold your breath for such a feature.

提交回复
热议问题