Python: Elegant and efficient ways to mask a list

Question

Example:

from __future__ import division
import numpy as np

n = 8
"""masking lists"""
lst = range(n)
print lst

# the mask (filter)
msk = [(el>3) and (el<=6) for el in lst]
print msk

# use of the mask
print [lst[i] for i in xrange(len(lst)) if msk[i]]

"""masking arrays"""
ary = np.arange(n)
print ary

# the mask (filter)
msk = (ary>3)&(ary<=6)
print msk

# use of the mask
print ary[msk]                          # very elegant  

and the results are:

>>> 
[0, 1, 2, 3, 4, 5, 6, 7]
[False, False, False, False, True, True, True, False]
[4, 5, 6]
[0 1 2 3 4 5 6 7]
[False False False False  True  True  True False]
[4 5 6]

As you see the operation of masking on array is more elegant compared to list. If you try to use the array masking scheme on list you'll get an error:

>>> lst[msk]
Traceback (most recent call last):
  File "<interactive input>", line 1, in <module>
TypeError: only integer arrays with one element can be converted to an index

The question is to find an elegant masking for lists.

Updates:
The answer by jamylak was accepted for introducing compress however the points mentioned by Joel Cornett made the solution complete to a desired form of my interest.

>>> mlist = MaskableList
>>> mlist(lst)[msk]
>>> [4, 5, 6]

Answer 1

You are looking for itertools.compress

Example from the docs

Equivalent to:

def compress(data, selectors):
    # compress('ABCDEF', [1,0,1,0,1,1]) --> A C E F
    return (d for d, s in izip(data, selectors) if s)

Answer 2

Since jamylak already answered the question with a practical answer, here is my example of a list with builtin masking support (totally unnecessary, btw):

from itertools import compress
class MaskableList(list):
    def __getitem__(self, index):
        try: return super(MaskableList, self).__getitem__(index)
        except TypeError: return MaskableList(compress(self, index))

Usage:

>>> myList = MaskableList(range(10))
>>> myList
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> mask = [0, 1, 1, 0]
>>> myList[mask]
[1, 2]

Note that compress stops when either the data or the mask runs out. If you wish to keep the portion of the list that extends past the length of the mask, you could try something like:

from itertools import izip_longest

[i[0] for i in izip_longest(myList, mask[:len(myList)], fillvalue=True) if i[1]]

Answer 3

If you are using Numpy, you can do it easily using Numpy array without installing any other library:

>> a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>> msk = [ True, False, False,  True,  True,  True,  True, False, False, False]
>> a = np.array(a) # convert list to numpy array
>> result = a[msk] # mask a
>> result.tolist()
[0, 3, 4, 5, 6]

Answer 4

i don't consider it elegant. It's compact, but tends to be confusing, as the construct is very different than most languages.

As Rossum has said about language design, we spend more time reading it than writing it. The more obscure the construction of a line of code, the more confusing it becomes to others, who may lack familiarity with Python, even though they have full competency in any number of other languages.

Readability trumps short form notations everyday in the real world of servicing code. Just like fixing your car. Big drawings with lots of information make troubleshooting a lot easier.

For me, I would much rather troubleshoot someone's code that uses the long form

print [lst[i] for i in xrange(len(lst)) if msk[i]]

than the numpy short notation mask. I don't need to have any special knowledge of a specific Python package to interpret it.