Python: Way to speed up a repeatedly executed eval statement?

Question

In my code, I\'m using eval to evaluate a string expression given by the user. Is there a way to compile or otherwise speed up this statement?

Answer 1

You can avoid the overhead by compiling the expression in advance using compiler.compile() for Python 2 or compile() for Python 3 :

In [1]: import math, compiler

In [2]: v = {'x': 2, 'y': 4}

In [3]: expression = "math.sin(v['x']) * v['y']"

In [4]: %timeit eval(expression)
10000 loops, best of 3: 19.5 us per loop

In [5]: compiled = compiler.compile(expression, '<string>', 'eval')

In [6]: %timeit eval(compiled)
1000000 loops, best of 3: 823 ns per loop

Just make sure you do the compiling only once (outside of the loop). As mentioned in comments, when using eval on user submitted strings make sure you are very careful about what you accept.

Answer 2

You can also trick python:

expression = "math.sin(v['x']) * v['y']"
exp_as_func = eval('lambda: ' + expression)

And then use it like so:

exp_as_func()

Speed test:

In [17]: %timeit eval(expression)
10000 loops, best of 3: 25.8 us per loop

In [18]: %timeit exp_as_func()
1000000 loops, best of 3: 541 ns per loop

---

As a side note, if v is not a global, you can create the lambda like this:

exp_as_func = eval('lambda v: ' + expression)

and call it:

exp_as_func(my_v)

Answer 3

I think you are optimising the wrong end. If you want to perform the same operation for a lot of numbers you should consider using numpy:

import numpy
import time
import math
import random

result_count = 100000
expression = "sin(x) * y"

namespace = dict(
    x=numpy.array(
        [random.random() for _ in xrange(result_count)]),
    y=numpy.array(
        [random.random() for _ in xrange(result_count)]),
    sin=numpy.sin,
)
print ('Evaluating %d instances '
       'of the given expression:') % result_count
print expression

start = time.time()
result = eval(expression, namespace)
numpy_time = time.time() - start
print "With numpy:", numpy_time


assert len(result) == result_count
assert all(math.sin(a) * b == c for a, b, c in
           zip(namespace["x"], namespace["y"], result))

To give you an idea about the possible gain I've added a variant using generic python and the lambda trick:

from math import sin
from itertools import izip

start = time.time()
f = eval("lambda: " + expression)
result = [f() for x, y in izip(namespace["x"], namespace["y"])]
generic_time = time.time() - start
print "Generic python:", generic_time
print "Ratio:", (generic_time / numpy_time)

Here are the results on my aging machine:

$ python speedup_eval.py 
Evaluating 100000 instances of the given expression:
sin(x) * y
With numpy: 0.006098985672
Generic python: 0.270224094391
Ratio: 44.3063992807

The speed-up is not as high as I expected, but still significant.