Removing the lists from a list which are duplicated for some items

Question

I\'m trying to remove the lists from a list which have same first and third items but only keeping the first one. Example list and output:

li=[ [2,4,5], [1,3         


        

Answer 1

This is a solution based on @iurisilvio's iterator comment and which uses itertools.compress in conjunction with the set-based solutions from the others. Instead of building up an output_list from elements in the input list, a selector list comprising Boolean values is built up on a one:one basis with respect to elements from the input list. A value of True indicates that the corresponding element from the input list should be retained in the output. The selector can then be applied on the input list via itertools.compress to product an output iterable.

from itertools import compress
li=[ [2,4,5], [1,3,5], [1,6,5] ]
b_li = set()
selectors = []
for x in li:
    s = (x[0], x[2])
    if s not in b_li:
        b_li.add(s)
        selectors.append(True)
    else:
        selectors.append(False)

for x in compress(li, selectors):
  print x
[2, 4, 5]
[1, 3, 5]

Answer 2

Leveraging OrderedDict and the fact that dictionaries have unique keys.

>>> from collections import OrderedDict
>>> li=[ [2,4,5], [1,3,5], [1,6,5] ]
>>> OrderedDict(((x[0], x[2]), x) for x in reversed(li)).values()
[[1, 3, 5], [2, 4, 5]]

Answer 3

Use a set for storing the seen elements. That is faster:

seen = set()
res = []
for entry in li:
    cond = (entry[0], entry[2])
    if cond not in seen:
        res.append(entry)
        seen.add(cond)


[[2, 4, 5], [1, 3, 5]]

ADDITION

Also, the time spend on thinking about telling variables names is typically well spend. Often things first though of as throw-away solutions stick around much longer than anticipated.

Answer 4

An improved version:

b_li = set()
output_list = []
b_li_add = b_li.add
output_list_append = output_list.append
for x in li:
    s = (x[0], x[2])
    if s not in b_li:
        b_li_add(s)
        output_list_append(x)

The changes are:

• Use a set() for b_li which makes lookups faster.
• Turn s into a tuple as there is no need to store unique first and third elements as lists.
• Reduced function lookups which speeds up the code as well.