Clojure head retention

Question

I\'m reading Clojure Programming book by O\'Reilly..

I came over an example of head retention. First example retains reference to d (I presume), so it d

Answer 1

Answer by @Michał Marczyk, while correct, is a little difficult to comprehend. I find this post on Google Groups easier to grasp.

Here's how I understand it:

Step 1 Create lazy sequence: (range 1e8). Values are not realized yet, I marked them as asterixes (*):

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * ... * * *

Step 2 Create two more lazy seqences which are "windows" through which you look at the original, huge lazy sequence. First window contains only 12 elements (t), the other the rest of elements (d):

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * ... * * *
t t t t t t t t t t t t t d d d d d d d d d d d d d d d d d ... d d d 

Step 3 - out of memory scenario - you evaluate [(count d) (count t)]. So, first you count elements in d, then in t. What will happen is that you will go through all values starting at the first element of d and realize them (marked as !):

* * * * * * * * * * * * * ! * * * * * * * * * * * * * * * * ... * * *
t t t t t t t t t t t t t d d d d d d d d d d d d d d d d d ... d d d 
                          ^
                         start here and move right ->

* * * * * * * * * * * * * ! ! * * * * * * * * * * * * * * * ... * * *
t t t t t t t t t t t t t d d d d d d d d d d d d d d d d d ... d d d 
                            ^

* * * * * * * * * * * * * ! ! ! * * * * * * * * * * * * * * ... * * *
t t t t t t t t t t t t t d d d d d d d d d d d d d d d d d ... d d d 
                              ^

                     ...

; this is theoretical end of counting process which will never happen
; because of OutOfMemoryError
* * * * * * * * * * * * * ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ... ! ! !
t t t t t t t t t t t t t d d d d d d d d d d d d d d d d d ... d d d 
                                                                    ^

Problem is that all the realized values (!) are being retained, because the head of the collection (first 12 elements) are still needed - we still need to evaluate (count t). This consumes a lot of memory causing JVM to crash.

Step 3 - valid scenario - this time you evaluate [(count t) (count d)]. So we first want to count elements in smaller, head sequence:

! * * * * * * * * * * * * * * * * * * * * * * * * * * * * * ... * * *
t t t t t t t t t t t t t d d d d d d d d d d d d d d d d d ... d d d 
^
start here and move right ->

                        ! * * * * * * * * * * * * * * * * * ... * * *
t t t t t t t t t t t t t d d d d d d d d d d d d d d d d d ... d d d 
                        ^

Then, we count elements in d sequence. Compiler knows that elements from t aren't needed anymore, so it can garbage collect them freeing up the memory:

                          ! * * * * * * * * * * * * * * * * ... * * *
t t t t t t t t t t t t t d d d d d d d d d d d d d d d d d ... d d d 
                          ^

                            ! * * * * * * * * * * * * * * * ... * * *
t t t t t t t t t t t t t d d d d d d d d d d d d d d d d d ... d d d 
                            ^

                     ...

                                                            ...     !
t t t t t t t t t t t t t d d d d d d d d d d d d d d d d d ... d d d 
                                                                    ^

Now we can see that, because elements from t weren't needed anymore, compiler was able to clear memory as it went through the large sequence.