What is the pythonic way of collapsing values into a set for multiple columns per each group in pandas dataframes?

问题

Given a dataframe, collapsing values into a set per group for a column is straightforward:

df.groupby('A')['B'].apply(set)

But how do you do it in a pythonic way if you want to do it on multiple columns and the result to be in a dataframe?

For example for the following dataframe:

import pandas as pd
df = pd.DataFrame({'user_id': [1, 2, 3, 4, 1, 2, 3], 
               'class_type': ['Krav Maga', 'Yoga', 'Ju-jitsu', 'Krav Maga', 'Ju-jitsu','Krav Maga', 'Karate'], 
               'instructor': ['Bob', 'Alice','Bob', 'Alice','Alice', 'Alice','Bob']})

The result wanted is the data frame below produced in a pythonic way:

|user_id|class_type             |instructor     |
|-------|-----------------------|---------------|
|  1    | {Krav Maga, Ju-jitsu} | {Bob, Alice}  |
|  2    | {Krav Maga, Yoga}     | {Alice}       | 
|  3    | {Karate, Ju-jitsu}    | {Bob}         | 
|  4    | {Krav Maga}           | {Alice}       |

This is a dummy example. The question spurred from: "what if I have a table with 30 columns and I want to achieve this in a pythonic way?"

Currently I have a solution but I don't think is the best way to do it:

df[['grouped_B', 'grouped_C']] = df.groupby('A')[['B','C']].transform(set)
deduped_and_collapsed_df = df.groupby('A')[['A','grouped_B', 'grouped_C']].head(1)

Thank you in advance!

回答1:

In [11]: df.groupby('user_id', as_index=False).agg(lambda col: set(col.values.tolist()))
Out[11]:
   user_id             class_type    instructor
0        1  {Krav Maga, Ju-jitsu}  {Alice, Bob}
1        2      {Yoga, Krav Maga}       {Alice}
2        3     {Ju-jitsu, Karate}         {Bob}
3        4            {Krav Maga}       {Alice}

or shorter version from @jezrael:

In [12]: df.groupby('user_id').agg(lambda x: set(x))
Out[12]:
                    class_type    instructor
user_id
1        {Krav Maga, Ju-jitsu}  {Alice, Bob}
2            {Yoga, Krav Maga}       {Alice}
3           {Ju-jitsu, Karate}         {Bob}
4                  {Krav Maga}       {Alice}

回答2:

Here is a collections.defaultdict method. Pythonic is subjective.

This solution is certainly not Pandonic / Pandorable. Dataframes generally have large overheads when using groupby.agg with lambda, so you might find the below solution more efficient.

from collections import defaultdict

d_class, d_instr = defaultdict(set), defaultdict(set)

for row in df.itertuples():
    idx, class_type, instructor, user_id = row
    d_class[user_id].add(class_type)
    d_instr[user_id].add(instructor)

res = pd.DataFrame([d_class, d_instr]).T.rename(columns={0: 'class_type', 1: 'instructor'})

Result:

              class_type    instructor
1  {Krav Maga, Ju-jitsu}  {Bob, Alice}
2      {Krav Maga, Yoga}       {Alice}
3     {Ju-jitsu, Karate}         {Bob}
4            {Krav Maga}       {Alice}

来源：https://stackoverflow.com/questions/49535966/what-is-the-pythonic-way-of-collapsing-values-into-a-set-for-multiple-columns-pe

标签

python

pandas

pandas-groupby