Python 3.5 vs. 3.6 what made “map” slower compared to comprehensions

Question

I sometimes used map if there was a function/method that was written in C to get a bit extra performance. However I recently revisited some of my benchmarks and

Answer 1

I think a fair comparison would involve using the same function. In the case of your example, when comparison is fair, map still wins:

>>> import sys
>>> print(sys.version)
3.6.2 |Continuum Analytics, Inc.| (default, Jul 20 2017, 13:14:59) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]
>>> import random
>>> lst = [random.randint(0, 10) for _ in range(100000)]
>>> assert list(map((5).__lt__, lst)) == [5 < i for i in lst]
>>> f = (5).__lt__
>>> %timeit list(map(f, lst))
4.63 ms ± 110 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
>>> %timeit [f(i) for i in lst]
9.17 ms ± 177 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

While in Python 3.5 (at least on my system) map is faster than in Python 3.6, so is list comprehension:

>>> print(sys.version)
3.5.3 |Continuum Analytics, Inc.| (default, Mar  6 2017, 12:15:08) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]
>>> %timeit list(map(f, lst))
100 loops, best of 3: 4.36 ms per loop
>>> %timeit [f(i) for i in lst]
100 loops, best of 3: 8.12 ms per loop

Still, when using the same function, map is ~2x faster than list comprehension in both Python 3.5 and 3.6.

EDIT (Response to @user2357112 comments):

It is my opinion that performing "fair" comparisons is important in answering OP's question: "My question is what happened in this case that made the list-comprehension faster and the map solution slower?" (last paragraph). However, in the first paragraph, @MSeifert says: "... [I] noticed that the relative performance (compared to a similar list comprehension) drastically changed between Python 3.5 and 3.6" That is, the comparison is between a map and list comprehension. Yet, @MSeifert tests are set-up as follows:

timig_map_35 = Timing(list(map(f, lst)))
timing_list_35 = Timing([g(i) for i in lst])

This kind of testing makes it difficult to find the cause of timing differences: are they because list comprehension became faster in 3.6 or map became slower in 3.6 or f(i) is slower in 3.6 or g(i) is faster in 3.6...

Therefore, I proposed to introduce f = (5).__lt__ and use the same function in both map and list comprehension tests. I have also modified @MSeifert test by increasing the number of elements in the list and redusing number of "loops" in the timeit:

import random
lst = [random.randint(0, 10) for _ in range(1000000)] # 10x more elements
f = (5).__lt__
%timeit -n1 -r1000 list(map(f, lst)) # f = (5).__lt__
%timeit -n1 -r1000 [f(i) for i in lst] # f(i) = (5).__lt__(i)
%timeit -n1 -r1000 [5 < i for i in lst] # g(i) = 5 < i
%timeit -n1 -r1000 [1 for _ in lst] # h(i) = 1

In Python 3.6 I get:

43.5 ms ± 1.79 ms per loop (mean ± std. dev. of 1000 runs, 1 loop each)
82.2 ms ± 2.39 ms per loop (mean ± std. dev. of 1000 runs, 1 loop each)
43.6 ms ± 1.64 ms per loop (mean ± std. dev. of 1000 runs, 1 loop each)
23.8 ms ± 1.27 ms per loop (mean ± std. dev. of 1000 runs, 1 loop each)

In Python 3.5 I get:

1 loop, best of 1000: 43.7 ms per loop
1 loop, best of 1000: 78.9 ms per loop
1 loop, best of 1000: 46 ms per loop
1 loop, best of 1000: 26.8 ms per loop

In my opinion this shows that list comprehension is slightly faster in 3.6 than in 3.5 except when f is used. Therefore, it is difficult to conclude that it is the map that is slower in Python 3.6 or is first timeit above is slower because of the call to f being slower. Therefore I performed two more tests:

%timeit -n1 -r1000 list(map(abs, lst))
%timeit -n1 -r1000 [abs(i) for i in lst]
%timeit -n1000000 -r1000 f(1)

In Python 3.6 I get:

25.8 ms ± 1.42 ms per loop (mean ± std. dev. of 1000 runs, 1 loop each)
67.1 ms ± 2.07 ms per loop (mean ± std. dev. of 1000 runs, 1 loop each)
64.7 ns ± 2.22 ns per loop (mean ± std. dev. of 1000 runs, 1000000 loops each)

In Python 3.5 I get:

1 loop, best of 1000: 38.3 ms per loop
1 loop, best of 1000: 56.4 ms per loop
1000000 loops, best of 1000: 59.6 ns per loop

This shows that map can be significantly faster than list comprehension for some functions: specifically, for abs(x) relative performance of map versus "list comprehension" in Python 3.6 is 67.1/25.8 = 2.60 while in Python 3.5 it is 56.4/38.3 = 1.47. Therefore it is interesting to know why @MSeifert test shows that map is slower in Python 3.6. My last test above shows timing test for f(1) "alone". I am not sure how valid this test is (unfortunately) - I wanted to avoid using map or [for] to eliminate one variable - but it shows that in Python 3.6 f = (5).__lt__ became slower than in Python 3.5. Therefore I conclude that it is the particular form of the function f ((5).__lt__) whose evaluation slowed and not the map function. I know this last "alone" test is likely a bad test, however, the fact that map is very fast (relatively or absolutely) when used with abs shows that the problem is in f and not in map.

NOTE: Python 3.5 uses IPython 5.3.0 and Python 3.6 uses IPython 6.1.0.