call-with-current-continuation - state saving concept

Question

After reading The Seasoned Schemer I felt I understood call/cc properly. But, after seeing some WOW tricks with call/cc I found I

Answer 1

Your suspicion that something is wrong is correct. The code is totally broken, and this is obvious from the fact that the generator fails to capture a new continuation for the mainline program whenever it is invoked to call the item. Or rather, it fumbles and throws that continuation away. The result is that the wrong continuation is called on the attempt to get the second item, resulting in an infinite loop.

Firstly, let's correct something from the wording of your question. Calling next doesn't yield items; calling next yields the generator function. The way next is supposed to be used is exemplified like this:

(let ((g (next)))
  (list (g) (g) (g)))  ;; should return (0 1 done)

But, in fact it cannot work. Let's examine it:

(define (itr lst)
  (define (state k)
    (for-each (lambda (item)
                (call/cc (lambda (h)
                           (set! state h)
                           (k item))))
              lst)
    (k 'done))

  (define (generator)
    (call/cc (lambda (k) (state k))))
  generator)

(define (next)
  (itr (range 2)))

Let's trace through what is happening.

The setup: When (next) is called, the expression (iter (range 2)) returns generator, a closure captured in an environment where the itr, lst and state variables are bound.

The first iteration: The first call to the generator returned by next therefore invokes generator. Now generator captures its own continuation, which appears as k in the lambda, and passes it to state. So then state runs, and has generator's continuation bound to k. It enters into the first iteration, and saves its own state by replacing itself with a new continuation: (set! state h). At this point, the previous binding of state to the define-d function is overwritten; state is now a continuation function to resume the for-each. The next step is to yield the item back to the k continuation, which brings us back to generator which returns the item. Great, so that's how the first item appears out of the first call to (next).

The second iteration: From here on, things go wrong. The second call to the generator that was returned by next again, captures a continuation again and invokes state which is now the continuation of the generating co-routine. The generator passes its own continuation into state. But state is no longer the function that was define-d by itr! And so the newly captured continuation in generator does not connect with the k parameter that is in the lexical scope of the for-each. When (k item) is called to yield the second item, this k still refers to the original k binding which holds the originally captured continuation in the first call to generator. This is analogous to a backwards goto and results in non-terminating behavior.

Here is how we can fix it:

(define (itr lst)
  (define yield '()) ;; forward definition (could use let for this).

  (define (state)    ;; k parameter is gone
    (for-each (lambda (item)
                (call/cc (lambda (h)
                           (set! state h)
                           (yield item))))  ;; call yield, not k
              lst)
    (yield 'done))  ;; yield, not k.

  (define (generator)
    (call/cc (lambda (self) 
               (set! yield self) ;; save new escape on each call
               (state))))
  generator)

;; test
(let ((g (itr (range 2))) ;; let's eliminate the "next" wrapper
  (display (list (g) (g) (g))))

The output is (0 1 done).