Remove duplicates from a list of dictionaries when only one of the key values is different

Question

I have seen some similar answers, but I can\'t find something specific for this case:

I have a list of dictionaries like this:

[
 {\"element\":Bla, \         


        

Answer 1

You say you have a lot of other keys in the dictionary not mentioned in the question.

Here is O(n) algorithm to do what you need:

>>> seen = set()
>>> result = []
>>> for d in dicts:
...     h = d.copy()
...     h.pop('date')
...     h = tuple(h.items())
...     if h not in seen:
...         result.append(d)
...         seen.add(h)

>>> pprint(result)
[{'date': '12/04/12', 'element': 'Bla', 'version': 2},
 {'date': '12/04/12', 'element': 'Bla', 'version': 3}]

h is a copy of the dict. date key is removed from it with pop.

Then tuple is created as a hashable type which can be added to set.

If h has never been seen before, we append it to result and add to seen. Additions to seen is O(1) as well as lookups (h not in seen).

At the end, result contains only unique elements in terms of defined h values.