Inconsistency between %time and %timeit in IPython

Question

I am confronted to a weird situation that I can't explain. Here is my test timing the generation of a large list of tuples:

In [1]: def get_list_of_tuples():
   ...:     return [(i,) for i in range(10**6)]
   ...:

In [2]: %time res = get_list_of_tuples()
CPU times: user 0.93 s, sys: 0.08 s, total: 1.01 s
Wall time: 0.98 s

In [3]: %timeit res = get_list_of_tuples()
1 loops, best of 3: 92.1 ms per loop

As you can see, the generation of this large list of tuples takes just below a second. timeit reports the execution time to be around 0.1 second. Why is there such a big difference in the two reports?

(Tested on IPython 0.11, Python 2.6.5.)

Answer 1

The main difference is because "by default, timeit() temporarily turns off garbage collection during the timing".

Turning the garbage collection returns results similar to the one shown in the question, i.e. the time of execution with garbage collection is magnitude bigger than the one without:

In [1]: import timeit

# Garbage collection on.
In [2]: N = 10; timeit.timeit('[(i,) for i in range(10**6)]', 'gc.enable()', number=N) / N
Out[2]: 0.74884700775146484
# 749 ms per loop.

# Garbage collection off.
In [3]: N = 10; timeit.timeit('[(i,) for i in range(10**6)]', number=N) / N
Out[3]: 0.15906109809875488
# 159 ms per loop.

Answer 2

Benoit,

If I use Python 2.6.6 and IPython 0.10 then I see similar answers to yours. Using Python 2.7.1 and IPython 0.10.1 I get something more sensible:

% ipython
Python 2.7.1 (r271:86832, Nov  3 2011, 16:23:57) 
Type "copyright", "credits" or "license" for more information.

IPython 0.10.1 -- An enhanced Interactive Python.

In [1]: def get_list_of_tuples():
   ...:     return [(i,) for i in range(10**6)]
   ...: 

In [2]: %time res = get_list_of_tuples()
CPU times: user 0.25 s, sys: 0.10 s, total: 0.35 s
Wall time: 0.35 s

In [3]: %timeit res = get_list_of_tuples()
1 loops, best of 3: 215 ms per loop

Answer 3

%time - runs statement only once, and have measurement error

%timeit - runs statement few times, and choses most accurate time.

See Python timeit module documentation for some explanations