How can I combine dictionaries with the same keys?

Question

I have a list of dictionaries like so:

dicts = [
    {\'key_a\': valuex1,
     \'key_b\': valuex2,
     \'key_c\': valuex3},

    {\'key_a\': valuey1,
     \         


        

Answer 1

You can merge dictionaries in the following way:

def merge_dicts(dict_list, separator=''):
    """
    Merges list of dictionaries to a single dictionary, Concatenates values with the same key.
    :param dict_list: list of dictionaries to be merged.
    :param separator: separator to be inserted between values of same key.
    :return: Merged dictionary.
    """
    return {k1: separator.join([d[k1] for d in dict_list if k1 in d])
            for k1 in set(reduce(lambda x, y: x+y, [k.keys() for k in dict_list]))
    }

Answer 2

If all the dicts have the same set of keys, this will work:

dict((k, [d[k] for d in dictList]) for k in dictList[0])

If they may have different keys, you'll need to first built a set of keys by doing set unions on the keys of the various dicts:

allKeys = reduce(operator.or_, (set(d.keys()) for d in dictList), set())

Then you'll need to protect against missing keys in some dicts:

dict((k, [d[k] for d in [a, b] if k in d]) for k in allKeys)

Answer 3

big_dict = {}
for k in dicts[0]:
    big_dict[k] = [d[k] for d in dicts]

Or, with a dict comprehension:

{k: [d[k] for d in dicts] for k in dicts[0]}

Answer 4

You can use collections.defaultdict. The benefit of this solution is it does not require keys to be consistent across dictionaries, and it still maintains the minimum O(n) time complexity.

from collections import defaultdict

dict_list = [{'key_a': 'valuex1', 'key_b': 'valuex2', 'key_c': 'valuex3'},
             {'key_a': 'valuey1', 'key_b': 'valuey2', 'key_c': 'valuey3'},
             {'key_a': 'valuez1', 'key_b': 'valuez2', 'key_c': 'valuez3'}]            

d = defaultdict(list)
for myd in dict_list:
    for k, v in myd.items():
        d[k].append(v)

Result:

print(d)

defaultdict(list,
            {'key_a': ['valuex1', 'valuey1', 'valuez1'],
             'key_b': ['valuex2', 'valuey2', 'valuez2'],
             'key_c': ['valuex3', 'valuey3', 'valuez3']})

Answer 5

If you are happy to use a 3rd party library you can use Pandas. The pd.DataFrame constructor accepts a list of dictionaries directly:

import pandas as pd

res = pd.DataFrame(dictList).to_dict(orient='list')

{'key_a': ['valuex1', 'valuey1', 'valuez1'],
 'key_b': ['valuex2', 'valuey2', 'valuez2'],
 'key_c': ['valuex3', 'valuey3', 'valuez3']}