doseq over a simple lazy seq runs out of heap space

六月ゝ 毕业季﹏ 提交于 2020-01-02 17:16:54

问题


When stress-testing some Clojure code at work, I noticed it runs out of heap space when iterating over large data-sets. I eventually managed to trace the issues back to the combination of Clojure's doseq function, and implementation fo lazy sequences.

This is the minimal code snippet that crashes Clojure by exhausting available heap space:

(doseq [e (take 1000000000 (iterate inc 1))] (identity e))

The documentation for doseq clearly states that it doesn't retain the head of the lazy sequence, so I would expect the memory complexity of the above code to be close to O(1). Is there something I'm missing? What's the Clojure-idiomatic way of iterating over extremely large lazy sequences, if doseq isn't up to the job?


回答1:


When I run this sample I see the memory usage hit 2.0 Gigs so perhaps you are actually just running out of ram.

it sure does take a while to run:

user=> (time (doseq [e (take 1000000000 (iterate inc 1))] (identity e)))
"Elapsed time: 266396.221132 msecs"

form top:

23999 arthur    20   0 4001m 1.2g 5932 S  213 15.3  17:11.35 java                                          
24017 arthur    20   0 3721m 740m 5548 S   88  9.3  13:49.95 java  


来源:https://stackoverflow.com/questions/11932900/doseq-over-a-simple-lazy-seq-runs-out-of-heap-space

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!