Generators vs List Comprehension performance in Python

Question

Currently I was learning about generators and list comprehension, and messing around with the profiler to see about performance gains stumbled into this cProfile of a sum of

Answer 1

First of all the calls are to next(or __next__ in Python 3) method of the generator object not for some even number check.

In Python 2 you are not going to get any additional line for a list comprehension(LC) because LC are not creating any object, but in Python 3 you will because now to make it similar to a generator expression an additional code object() is created for a LC as well.

>>> cProfile.run('sum([number for number in range(9999999) if number % 2 == 0])')
         5 function calls in 1.751 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    1.601    1.601    1.601    1.601 :1()
        1    0.068    0.068    1.751    1.751 :1()
        1    0.000    0.000    1.751    1.751 {built-in method exec}
        1    0.082    0.082    0.082    0.082 {built-in method sum}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

>>> cProfile.run('sum((number for number in range(9999999) if number % 2 == 0))')
         5000005 function calls in 2.388 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
  5000001    1.873    0.000    1.873    0.000 :1()
        1    0.000    0.000    2.388    2.388 :1()
        1    0.000    0.000    2.388    2.388 {built-in method exec}
        1    0.515    0.515    2.388    2.388 {built-in method sum}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

The number of calls are different though 1(LC) compared to 5000001 in generator expression, this is most because sum is consuming the iterator hence has to call its __next__ method 500000 + 1 times(last 1 is probably for StopIteration to end the iteration). For a list comprehension all the magic happens inside its code object where the LIST_APPEND helps it in appending items one by one to the list, i.e no visible calls for cProfile.