R and shared memory for parallel::mclapply

前端 未结 3 1647
死守一世寂寞
死守一世寂寞 2020-12-16 20:20

I am trying to take advantage of a quad-core machine by parallelizing a costly operation that is performed on a list of about 1000 items.

I am using R\'s parallel::m

3条回答
  •  情歌与酒
    2020-12-16 21:04

    Just the tip what might have been going on R-devel Digest, Vol 149, Issue 22

    Radford Neal's answer from Jul 26, 2015:

    When mclapply forks to start a new process, the memory is initially shared with the parent process. However, a memory page has to be copied whenever either process writes to it. Unfortunately, R's garbage collector writes to each object to mark and unmark it whenever a full garbage collection is done, so it's quite possible that every R object will be duplicated in each process, even though many of them are not actually changed (from the point of view of the R programs).

提交回复
热议问题