GroupBy two columns with margins for first level

问题

I am grouping a dataframe by 2 columns and i aggregate by the sum of the other columns. How I can have a total by the first grouped column in the same data frame?

for example my data frame is:

np.random.seed(0)
df = pd.DataFrame({'A' : ['foo', 'bar', 'foo', 'bar', 'foo', 'bar', 'foo', 'foo'],
               'B' : ['one', 'one', 'two', 'three', 'two', 'two', 'one', 'three'],
               'C' : np.random.randn(8),
               'D' : np.random.randn(8)})

The result of:

grouped = df.groupby(by=['A', 'B']).sum()

is:

                  C         D
A   B                        
bar one    0.400157  0.410599
    three  2.240893  1.454274
    two   -0.977278  0.121675
foo one    2.714141  0.340644
    three -0.151357  0.333674
    two    2.846296  0.905081

I what to get:

                  C         D
A   B                        
bar one    0.400157  0.410599
    two   -0.977278  0.121675
    three  2.240893  1.454274
    total  1.663773  1.986547
foo one    2.714141  0.340644
    two    2.846296  0.905081
    three -0.151357  0.333674
    total  5.409080  1.579400

how it can be done?

UPDATE: I found a similar question at Pandas groupby and sum total of group It has 2 more answer for this question.

回答1:

You can get clever with pd.Categorical to create a placeholder for "total" in the groupby output. This'll make it easy to compute and assign the total back to the result.

df.B = pd.Categorical(
         df.B, categories=np.append(df.B.unique(), 'total'))
v = df.groupby(by=['A', 'B']).sum()
v.loc(axis=0)[pd.IndexSlice[:,'total']] = v.groupby(level=0).sum().values

print(v)
                  C         D
A   B                        
bar one    0.400157  0.410599
    two   -0.977278  0.121675
    three  2.240893  1.454274
    total  1.663773  1.986547
foo one    2.714141  0.340644
    two    2.846296  0.905081
    three -0.151357  0.333674
    total  5.409080  1.579400

If you need to aggregate on different metrics:

df.B = pd.Categorical(
         df.B, categories=np.append(df.B.unique(), 'total'))
idx = pd.MultiIndex.from_product([df.A.unique(), df.B.cat.categories]) 

v = df.groupby(by=['A', 'B']).agg(['sum', 'count']).reindex(idx)
v.loc(axis=0)[pd.IndexSlice[:,'total']] = v.groupby(level=0, sort=False).sum().values

print(v)
                  C               D      
                sum count       sum count
foo one    2.714141   2.0  0.340644   2.0
    two    2.846296   2.0  0.905081   2.0
    three -0.151357   1.0  0.333674   1.0
    total  5.409080   5.0  1.579400   5.0
bar one    0.400157   1.0  0.410599   1.0
    two   -0.977278   1.0  0.121675   1.0
    three  2.240893   1.0  1.454274   1.0
    total  1.663773   3.0  1.986547   3.0

Another alternative is pivot_table which makes margin generation easier (although does not provide sub-level margins):

df.pivot_table(index=['A', 'B'], 
               values=['C', 'D'], 
               aggfunc=['sum', 'count'], 
               margins=True)

                sum           count     
                  C         D     C    D
A   B                                   
bar one    0.400157  0.410599   1.0  1.0
    two   -0.977278  0.121675   1.0  1.0
    three  2.240893  1.454274   1.0  1.0
foo one    2.714141  0.340644   2.0  2.0
    two    2.846296  0.905081   2.0  2.0
    three -0.151357  0.333674   1.0  1.0
All        7.072852  3.565947   8.0  8.0

来源：https://stackoverflow.com/questions/53719607/groupby-two-columns-with-margins-for-first-level

标签

python

pandas

dataframe

group-by

pandas-groupby