List comprehension, map, and numpy.vectorize performance

Question

I have a function foo(i) that takes an integer and takes a significant amount of time to execute. Will there be a significant performance difference between any of the foll

Answer 1

First comment: don't mix usage of xrange() or range() in your samples... doing so invalidates your question as you're comparing apples and oranges.

I second @Gabe's notion that if you have many large data structures, numpy should win overall... just keep in mind most of the time C is faster than Python, but then again, most of the time, PyPy is faster than CPython. :-)

As far as listcomps vs. map() calls go... one makes 101 function calls while the other one makes 102. meaning you won't see a significant difference in timing, as shown below using the timeit module as @Mike has suggested:

• List Comprehension

  $ python -m timeit "def foo(x):pass; [foo(i) for i in range(100)]"
1000000 loops, best of 3: 0.216 usec per loop
$ python -m timeit "def foo(x):pass; [foo(i) for i in range(100)]"
1000000 loops, best of 3: 0.21 usec per loop
$ python -m timeit "def foo(x):pass; [foo(i) for i in range(100)]"
1000000 loops, best of 3: 0.212 usec per loop
  

• map() function call

  $ python -m timeit "def foo(x):pass; map(foo, range(100))"
1000000 loops, best of 3: 0.216 usec per loop
$ python -m timeit "def foo(x):pass; map(foo, range(100))"
1000000 loops, best of 3: 0.214 usec per loop
$ python -m timeit "def foo(x):pass; map(foo, range(100))"
1000000 loops, best of 3: 0.215 usec per loop
  

With that said however, unless you are planning on using the lists that you create from either of these techniques, try avoid them (using lists) completely. IOW, if all you're doing is iterating over them, it's not worth the memory consumption (and possibly creating a potentially massive list in memory) when you only care to look at each element one at a time just discard the list as soon as you're done.

In such cases, I highly recommend the use of generator expressions instead as they don't create the entire list in memory... it is a more memory-friendly, lazy iterative way of looping through elements to process w/o creating a largish array in memory. The best part is that its syntax is nearly identical to that of listcomps:

a = (foo(i) for i in range(100))

2.x users only: along the lines of more iteration, change all the range() calls to xrange() for any older 2.x code then switch to range() when porting to Python 3 where xrange() replaces and is renamed to range().