Memoize decorators failing to memoize (when not using decorator syntax)

Question

I've got a simple memoizer decorator:

def funcmemo(f):
    memo = {}
    @wraps(f)
    def wrapper(*args):
    if args in memo:
        return memo[args]
    else:
        temp = f(*args)
        print "memoizing: ", args, temp  
        memo[args] = temp
        return temp
    return wrapper

Now, when I use it via the "@" token,

@funcmemo
def fib(n):
    print "fib called with:", n
    if n < 2: return n
    return fib(n-2) + fib(n-1)

res = fib(3)
print "result:", res

it works correctly, as seen in the printed output:

fib called with: 3
fib called with: 1
memoizing:  (1,) 1
fib called with: 2
fib called with: 0
memoizing:  (0,) 0
memoizing:  (2,) 1
memoizing:  (3,) 2
result:  2

However, when I do this:

def fib(n):
    print "fib called with:", n
    if n < 2: return n
    return fib(n-2) + fib(n-1)

memfib = funcmemo(fib)
res = memfib(3)
print "result:", res

Apparently an undecorated fib gets called, with only the final return value "reaching" the cache (obviously resulting in huge slowdown):

fib called with: 3
fib called with: 1
fib called with: 2
fib called with: 0
fib called with: 1
memoizing:  (3,) 2
result: 2

Curiously, this one works fine:

def fib(n):
    print "fib called with:", n
    if n < 2: return n
    return fib(n-2) + fib(n-1)

fib = funcmemo(fib)
res = fib(3)
print "result:", res

Also, the very same thing happens with a class-based version:

class Classmemo(object):
    def __init__ (self, f):
        self.f = f
        self.mem = {}
    def __call__ (self, *args):
        if args in self.mem:
            return self.mem[args]
        else:
            tmp = self.f(*args)
            print "memoizing: ", args, temp
            self.mem[args] = tmp
            return tmp

The problem also occurs when using an "anonymous" decorated function, like

res = Classmemo(fib)(3)

I'd be glad to be enlightened about the reasons behind this.

Answer 1

There is nothing curious about this. When you do

memofib = funcmemo(fib)

You're not changing the function fib points to in any way, but creating a new function and pointing the name memofib at it.

So when memofib gets called, it calls the function pointed to by the name fib -- which recursively calls itself, not memofib -- so no memoization occurs.

In your second example, you do

fib = funcmemo(fib)

so it calls itself recursively and memoization happens at all levels.

If you don't want to overwrite the name fib, as the decorator version or your second example does, you could alter fib to take a function name:

def fib(n, fibfunc):
    print "fib called with:", n
    if n < 2: return n
    return fibfunc(n-2, fibfunc) + fibfunc(n-1, fibfunc)

memofib = funcmemo(fib)
res = fib(3, memofib)

You could also use a fixed point combinator to avoid passing fibfunc every time:

def Y(f):
    def Yf(*args):
        return f(Yf)(*args)
    return f(Yf)

@Y
def fib(f):
    def inner_fib(n):
        print "fib called with:", n
        if n < 2: return n
        return f(n-2) + f(n-1)
    return inner_fib

Answer 2

In case your question is just a simple why, I guess the answer is just because the recursion-call with fib() does call the function with the name fib(). To decorate that you have to replace the value of the pointer fib; this is not done by memfib = funcmemo(fib) nor by the class version.