How (in)efficient is a list comprehension if you don't assign it?

Question

In this question, I\'m having an argument with a commenter who argues that

for t in threads:
    t.join()

would be better than

Answer 1

A list comprehension will always produce a list object, in this case with the return values of all the t.join() calls. Python thus produces as list with None values of length len(threads) for you. Python will never try to optimize away the list object creation.

Using map() is also not any more efficient as you add additional stack pushes with the lambda. Just stick with the explicit for loop.

Really, for a series of thread joins there is no point in trying to micro optimize here. You are hurting readability for a non-critical piece of code.

In other words, I entirely agree with the commenter. Do not use a list comprehension or map() just for the side effects and saving yourself having to hit ENTER and create two lines of code.

Quoting the Zen of Python:

• Readability counts.

Answer 2

If you want a one-liner for this, write an exhaust function, and use it in conjunction with a generator expression:

def exhaust(iterator):
    for _ in iterator:
        pass

exhaust(t.join() for t in threads)

Then you aren't paying the cost for all of the list storage.

Feel free to rename exhaust something more snappy or relevant to your usage.

I've seen this list-comprehension abuse quite a lot, even in interview coding samples where we've explicitly instructed candidates to avoid unbounded memory growth when solving the problem.

Answer 3

You can make a foreach that works as you want. I would recommend against it, as it's not a normal python way of doing things. Untested, but along the lines of this:

class foreach:
    def __init__(self, object_list):
        self.__object_list = object_list

    def __getattr__(self, name):
        def f(*args, **kwargs):
            for obj in self.__object_list:
                getattr(obj, name)(*args, **kwargs)
        return f


foreach(thread_list).join()

It pretty much creates a sort of proxy object that forwards any call with any parameters to all objects.

(I will most likely have a good chat if you if you would do this in my development team, but it does serve as an example of what is possible in python)

Answer 4

If you want a one-liner why not just do for t in threads: t.join()?

Seems like the simplest solution to me, and likely to be the fastest too (though really, the overhead of joining threads is likely to dwarf anything else IMO)