Upsample in pandas multi-index

问题

I am trying to upsample within a grouped DataFrame but am unsure how to get it to only upsample within the groups. I have a DataFrame that looks like:

cat      weekstart                  date      
0.0      2016-07-04 00:00:00+00:00  2016-07-04    1
                                    2016-07-06    1
                                    2016-07-07    2
         2016-08-15 00:00:00+00:00  2016-08-16    1
                                    2016-08-19    1
         2016-09-19 00:00:00+00:00  2016-09-20    1
                                    2016-09-21    1
         2016-12-19 00:00:00+00:00  2016-12-19    1
                                    2016-12-21    1

1.0      2016-07-25 00:00:00+00:00  2016-07-26    2
         2016-08-01 00:00:00+00:00  2016-08-03    1
         2016-08-08 00:00:00+00:00  2016-08-12    1

If I do something like df.unstack().fillna(0).stack() leads to:

cat      weekstart                  date      
0.0      2016-07-04 00:00:00+00:00  2016-1-1      0 
                                           .
                                           .
                                           .
                                    2016-07-04    1
                                    2016-07-06    1
                                    2016-07-07    2

because the minimum in the date column is 2016-1-1. What i'm after though is only sampling business days within each 'cat' and 'weekstart', like:

 cat      weekstart                  date      
 0.0      2016-07-04 00:00:00+00:00  2016-07-04    1
                                     2016-07-05    0 
                                     2016-07-06    1
                                     2016-07-07    2
                                     2016-07-8     0
          2016-08-15 00:00:00+00:00  2016-08-15    0
                                     2016-08-16    1
                                     2016-08-17    0
                                     2016-08-18    0
                                    2016-08-19    1

I've tried using:

 level_values = df.index.get_level_values
 df.groupby(
            [level_values(i) for i in [0, 1]] + [pd.Grouper('B', level=-1)]
            )
    .sum()

but it isn't working as expected.

回答1:

I think you need custom function with reindex by MultiIndex created by bdate_range:

def f(x):
    lvl0 = x.index.get_level_values(0)[0]
    lvl1 = x.index.get_level_values(1)[0]
    lvl2 = pd.bdate_range(start=lvl1, periods=5)
    mux = pd.MultiIndex.from_product([[lvl0], [lvl1], lvl2], names=x.index.names)
    return (x.reindex(mux, fill_value=0))

s1 = s.groupby(['cat','weekstart'], group_keys=False).apply(f)

print (s1)

cat  weekstart   date      
0.0  2016-07-04  2016-07-04    1
                 2016-07-05    0
                 2016-07-06    1
                 2016-07-07    2
                 2016-07-08    0
     2016-08-15  2016-08-15    0
                 2016-08-16    1
                 2016-08-17    0
                 2016-08-18    0
                 2016-08-19    1
     2016-09-19  2016-09-19    0
                 2016-09-20    1
                 2016-09-21    1
                 2016-09-22    0
                 2016-09-23    0
     2016-12-19  2016-12-19    1
                 2016-12-20    0
                 2016-12-21    1
                 2016-12-22    0
                 2016-12-23    0
1.0  2016-07-25  2016-07-25    0
                 2016-07-26    2
                 2016-07-27    0
                 2016-07-28    0
                 2016-07-29    0
     2016-08-01  2016-08-01    0
                 2016-08-02    0
                 2016-08-03    1
                 2016-08-04    0
                 2016-08-05    0
     2016-08-08  2016-08-08    0
                 2016-08-09    0
                 2016-08-10    0
                 2016-08-11    0
                 2016-08-12    1
Name: a, dtype: int64

Setup:

d = {(0.0, pd.Timestamp('2016-07-04 00:00:00'), pd.Timestamp('2016-07-07 00:00:00')): 2, (1.0, pd.Timestamp('2016-07-25 00:00:00'), pd.Timestamp('2016-07-26 00:00:00')): 2, (0.0, pd.Timestamp('2016-08-15 00:00:00'), pd.Timestamp('2016-08-16 00:00:00')): 1, (0.0, pd.Timestamp('2016-07-04 00:00:00'), pd.Timestamp('2016-07-04 00:00:00')): 1, (0.0, pd.Timestamp('2016-09-19 00:00:00'), pd.Timestamp('2016-09-20 00:00:00')): 1, (0.0, pd.Timestamp('2016-09-19 00:00:00'), pd.Timestamp('2016-09-21 00:00:00')): 1, (0.0, pd.Timestamp('2016-12-19 00:00:00'), pd.Timestamp('2016-12-19 00:00:00')): 1, (1.0, pd.Timestamp('2016-08-08 00:00:00'), pd.Timestamp('2016-08-12 00:00:00')): 1, (0.0, pd.Timestamp('2016-07-04 00:00:00'), pd.Timestamp('2016-07-06 00:00:00')): 1, (1.0, pd.Timestamp('2016-08-01 00:00:00'), pd.Timestamp('2016-08-03 00:00:00')): 1, (0.0, pd.Timestamp('2016-12-19 00:00:00'), pd.Timestamp('2016-12-21 00:00:00')): 1, (0.0, pd.Timestamp('2016-08-15 00:00:00'), pd.Timestamp('2016-08-19 00:00:00')): 1}
s = pd.Series(d).rename_axis(['cat','weekstart','date'])    
print (s)
cat  weekstart   date      
0.0  2016-07-04  2016-07-04    1
                 2016-07-06    1
                 2016-07-07    2
     2016-08-15  2016-08-16    1
                 2016-08-19    1
     2016-09-19  2016-09-20    1
                 2016-09-21    1
     2016-12-19  2016-12-19    1
                 2016-12-21    1
1.0  2016-07-25  2016-07-26    2
     2016-08-01  2016-08-03    1
     2016-08-08  2016-08-12    1
dtype: int64

来源：https://stackoverflow.com/questions/48665651/upsample-in-pandas-multi-index

标签

python

pandas

pandas-groupby