Python: remove duplicate items from a list while iterating

问题

I have a list named results, I want to get rid of the duplicate items in the list, the expected result and my results does not match, I checked my code but cannot find what is the problem, what happened? Thanks a lot!

results = [[-4, -2, 6], [-4, -2, 6], [-4, -2, 6], [-4, -2, 6], [-4, -2, 
    6], [-4, -2, 6], [-4, 0, 4], [-4, 0, 4], [-4, 1, 3], [-4, 1, 3], [-4, 2, 
    2], [-4, 2, 2], [-4, 2, 2], [-2, -2, 4], [-2, -2, 4], [-2, -2, 4], [-2, 
    -2, 4], [-2, 0, 2], [-2, 0, 2], [-2, 0, 2], [-2, -2, 4], [-2, -2, 4], 
    [-2, 0, 2], [-2, 0, 2], [-2, 0, 2], [-2, 0, 2], [-2, 0, 2], [-2, 0, 2]]

i = 0
while i < len(results):  
    j = i+1
    while j < len(results):
        if(set(results[i]) == set(results[j])):
            results.remove(results[j])
        else:
            j = j+1
    i = i+1
print(results)

OUTPUT:
[[-4,-2,6],[-4,0,4],[-4,1,3],[-4,2,2],[-2,-2,4],[-2,-2,4],[-2,0,2]]

EXPECTED RESULT:
[[-4,-2,6],[-4,0,4],[-4,1,3],[-4,2,2],[-2,-2,4],[-2,0,2]]

UPDATE: I got it. no problem with the logic of this code but I made a simple mistake with one place (sorry...I am a newbie). I should replace method "remove" by method "del", because I want to remove the item with specified index, if use "remove", it always remove the first one shows up in the list of that value. Anyway, Thanks to all!

For example:
myList = ['Apple', 'Banana', 'Carrot','Apple']
myList.remove(myList[3])
print(myList)
expected output:['Apple', 'Banana', 'Carrot']
actual output: ['Banana', 'Carrot', 'Apple']

myList = ['Apple', 'Banana', 'Carrot','Apple']
del (myList[3])
print(myList)
OUTPUT: ['Apple', 'Banana', 'Carrot']

SOLUTION to my question:

### use "del" instead of "remove"
#results.remove(results[j])
del results[j]

Another simple test example similar to my original question:
results = [[-2, -2, 4], [-2, 0, 2], [-2, 0, 2], [-2, 0, 2], [-2, -2, 4]]
i = 0
while i < len(results):  
    j = i+1
    while j < len(results):
        print(results[i],results[j])
            if(set(results[i]) == set(results[j])):
                #would get incorrect answer with "replace"
                results.remove(results[j])
                #try "del" to get the correct answer
                #del (results[j]) 
    else:
        j = j+1
i = i+1
print(results)

回答1:

Thanks to Remove duplicate sublists from a list

results = [[-4, -2, 6], [-4, -2, 6], [-4, -2, 6], [-4, -2, 6], [-4, -2, 
    6], [-4, -2, 6], [-4, 0, 4], [-4, 0, 4], [-4, 1, 3], [-4, 1, 3], [-4, 2, 
    2], [-4, 2, 2], [-4, 2, 2], [-2, -2, 4], [-2, -2, 4], [-2, -2, 4], [-2, 
    -2, 4], [-2, 0, 2], [-2, 0, 2], [-2, 0, 2], [-2, -2, 4], [-2, -2, 4], 
    [-2, 0, 2], [-2, 0, 2], [-2, 0, 2], [-2, 0, 2], [-2, 0, 2], [-2, 0, 2]]

results = [list(x) for x in set([tuple(x) for x in results])]
print (results)

prints:

[[-4, -2, 6], [-2, 0, 2], [-4, 1, 3], [-2, -2, 4], [-4, 2, 2], [-4, 0, 4]]

回答2:

The following loop will iterate backwards over the pairs, allowing you to mutate the list during iteration.

i = len(results) - 1
while i >= 1:  
    j = i - 1
    while j >= 0:
        if(set(results[i]) == set(results[j])):
            results.remove(results[j])
            i -= 1
        j -= 1
    i -= 1

But, yes, using sets would probably be better. A functional version of the above would be:

map(list, set(map(tuple, results)))

回答3:

Set-ify everything.

Convert the inner lists to sets to remove duplicate elements within them. Push all these sets into another set to get rid of duplicate (sub)sets. Finally, convert everything back to a list of lists:

no_dups = [list(s) for s in set(frozenset(l) for l in results))]

I refered to How can I create a Set of Sets in Python? to figure out the bit about frozenset. You can't just convert the sublists to sets since the outer set needs immutable elements to hash (and regular sets are mutable). Of course, you could get around this by using maps/tuples as other answers point out.

回答4:

You may use set with map as:

my_uniques = set(map(tuple, results))
#                     ^ type-cast inner list to tuples
#                       because lists are not hashable and
#                       hence can't be used with set

where my_uniques will be a set of unique tuple objects:

set([(-2, -2, 4), (-4, 0, 4), (-4, -2, 6), (-4, 2, 2), (-2, 0, 2), (-4, 1, 3)])

If you want to convert it to list of lists (which I don't think is necessary), you have to explicitly do:

my_uniques  = list(map(list, my_uniques))

来源：https://stackoverflow.com/questions/43529348/python-remove-duplicate-items-from-a-list-while-iterating