Clear R memory using Rpy2

问题

I have a bunch of R functions which I need to call through python. However, I reach memory errors when I try to allocate a large matrix. The same functions run fine on RStudio on the same computer. Here is a code chunk which crashes:

#python:
import rpy2.robjects as ro 
import gc
gc.collect()
ro.r.source("calibration_functions.R")
result1 = ro.r.func1()  #Does some calculations, works fine.
result2 = ro.r.func2(result1) #Crashes at this step

#R code:
func2 <- function(result1){
  preds_mat = matrix(data=NA, nrow = 263310, ncol = 1000)
  # do something...
  return(preds_mat)
}

The error I get is: RRuntimeError: Error: cannot allocate vector of size 1004.4 Mb

How can I clean the R memory? gc() or gc.collect() doesn't work.

回答1:

To clean the R memory:

rm(list = ls())

回答2:

(...)

The same functions run fine on RStudio on the same computer.

May be the same function, but likely with differences in memory usage from other applications.

Your R function func2() returns the following object size:

> object.size(func2(1))
1053240200 bytes

This is about 1.05Gb.

The error I get is: RRuntimeError: Error: cannot allocate vector of size 1004.4 Mb

The error observed is likely occurring either because of what is happening in the unspecified function func1(), or because something has changed between the execution in RStudio and the execution in rpy2.

回答3:

I typically deal with the issue by allocating more memory

from rpy2 import robjects
R = robjects.r


R('memory.limit()')
R('memory.limit(size = 10000)') ## in MB
R('memory.limit()')

…
R('gc()')## trigger garbage collection

来源：https://stackoverflow.com/questions/47399682/clear-r-memory-using-rpy2

标签

python

rpy2