Why doesn't python take advantage of __iadd__ for sum and chained operators?

问题

I just conducted an interesting test:

~$ python3 # I also conducted this on python 2.7.6, with the same result
Python 3.4.0 (default, Apr 11 2014, 13:05:11) 
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class Foo(object):
...     def __add__(self, other):
...         global add_calls
...         add_calls += 1
...         return Foo()
...     def __iadd__(self, other):
...         return self
...
>>> add_calls = 0
>>> a = list(map(lambda x:Foo(), range(6)))
>>> a[0] + a[1] + a[2]
<__main__.Foo object at 0x7fb588e6c400>
>>> add_calls
2
>>> add_calls = 0
>>> sum(a, Foo())
<__main__.Foo object at 0x7fb588e6c4a8>
>>> add_calls
6

Obviously, the __iadd__ method is more efficient than the __add__ method, not requiring the allocation of a new class. If my objects being added were sufficiently complicated, this would create unnecessary new objects, potentially creating huge bottlenecks in my code.

I would expect that, in an a[0] + a[1] + a[2], the first operation would call __add__, and the second operation would call __iadd__ on the newly created object.

Why doesn't python optimize this?

回答1:

The __add__ method is free to return a different type of object, while __iadd__ should, if using in-place semantics, return self. They are not required to return the same type of object here, so sum() should not rely on the special semantics of __iadd__.

You can use the functools.reduce() function to implement your desired functionality yourself:

from functools import reduce

sum_with_inplace_semantics = reduce(Foo.__iadd__, a, Foo())

Demo:

>>> from functools import reduce
>>> class Foo(object):
...     def __add__(self, other):
...         global add_calls
...         add_calls += 1
...         return Foo()
...     def __iadd__(self, other):
...         global iadd_calls
...         iadd_calls += 1
...         return self
... 
>>> a = [Foo() for _ in range(6)]
>>> result = Foo()
>>> add_calls = iadd_calls = 0
>>> reduce(Foo.__iadd__, a, result) is result
True
>>> add_calls, iadd_calls
(0, 6)

回答2:

Martjin's answer provides an excellent workaround, but I feel the need to summarize the bits and pieces of answers scattered throughout the comments:

The sum function is primarily used for immutable types. Performing all additions except the first in-place would create a performance improvement on objects that had an __iadd__ method, but checking for the __iadd__ method would cause a performance loss in the more typical case. Special cases aren't special enough to break the rules.

I also stated that __add__ should probably only be called once in a + b + c, where a + b creates a temporary variable, and then calls tmp.__iadd__(c) before returning it. However, this would violate the principle of least surprise.

回答3:

Since you are writting your class anyway, you know it's __add__ can return the same object as well, don't you?

And therefore you can do your currying optimized code to run with both the + operator and the built-in sum:

>>> class Foo(object):
...     def __add__(self, other):
...         global add_calls
...         add_calls += 1
...         return self

(Just beware of passing your code to third party functions that expect "+" to be a new object)

来源：https://stackoverflow.com/questions/31054393/why-doesnt-python-take-advantage-of-iadd-for-sum-and-chained-operators