How do I efficiently filter computed values within a Python list comprehension?

Question

The Python list comprehension syntax makes it easy to filter values within a comprehension. For example:

result = [x**2 for x in mylist if type(x) is int]
<         


        

Answer 1

If the calculations are already nicely bundled into functions, how about using filter and map?

result = filter (None, map (expensive, mylist))

You can use itertools.imap if the list is very large.

Answer 2

The most obvious (and I would argue most readable) answer is to not use a list comprehension or generator expression, but rather a real generator:

def gen_expensive(mylist):
    for item in mylist:
        result = expensive(item)
        if result:
            yield result

It takes more horizontal space, but it's much easier to see what it does at a glance, and you end up not repeating yourself.

Answer 3

You could always memoize the expensive() function so that calling it the second time around is merely a lookup for the computed value of x.

Here's just one of many implementations of memoize as a decorator.