Python: is there a way to import a variable using timeit.timeit()?

Question

Suppose I have some function that takes an array and changes every element to be 0.

def function(array):
    for i in range(0,len(array)):
        array[i] = 0
return array

I want to test how long this function takes to run on a random array, which I wish to generate OUTSIDE of the timeit test. In other words, I don't want to include the time it takes to generate the array into the time.

I first store a random array in a variable x and do:

timeit.timeit("function(x)",setup="from __main__ import function")

But this gives me an error: NameError: global name 'x' is not defined

How can I do this?

Answer 1

Import x from __main__ as well:

timeit.timeit("function(x)", setup="from __main__ import function, x")

Just like function, x is a name in the __main__ module, and can be imported into the timeit setup.

Answer 2

You can avoid this problem entirely if you pass timeit a function instead of a string. In that case, the function executes in its normal globals and closure environment. So:

timeit.timeit(lambda: function(x))

Or, if you prefer:

timeit.timeit(partial(function, x))

(See here for details. Note that it requires Python 2.6+, so if you need 2.3-2.5, you can't use this trick.)

---

As the documentation says, "Note that the timing overhead is a little larger in this case because of the extra function calls."

This means that it makes timeit itself run slower. For example:

>>> def f(): pass
>>> timeit.timeit('timeit.timeit("f()", setup="from __main__ import f")', setup='import timeit', number=1000)
91.66315175301861
>>> timeit.timeit(lambda: timeit.timeit(f), number=100)
94.89793294097762

However, it doesn't affect the actual results:

>>> timeit.timeit(f, number=100000000)
8.81197881908156
>>> timeit.timeit('f()', setup='from __main__ import f', number=100000000)
8.893913001054898

(In the rare cases where it does, that typically means one version or the other wasn't testing the function the way it will be called in your real code, or was testing the wrong closure or similar.)

Note that the actual time taken inside the function here is about 88 seconds, so we've nearly doubled the overhead of timing code… but it's still only added 3% to the total testing time. And the less trivial f is, the smaller this difference will be.

Answer 3

Import x from __main__:

timeit.timeit("function(x)",setup="from __main__ import function, x")

Answer 4

Alternatively, you can add x to globals. The good thing about it is that it works in a pdb debugging session:

globals()['x'] = x
timeit.timeit(lambda: function(x))

Note that the timing overhead is a little larger in this case because of the extra function calls. [source]

Answer 5

With Python 3.5, the optional argument globals has been introduced. It makes it possible to specify the namespace in which the timeit statement shall be executed.

So instead of writing:

timeit.timeit("function(x), setup="from __main__ import function, x")

...you can now write:

timeit.timeit("function(x)", globals=globals())