pandas-groupby

Python Pandas: Calculate moving average within group

主宰稳场 提交于 2020-03-18 04:36:06
问题 I have a dataframe containing time series for 100 objects: object period value 1 1 24 1 2 67 ... 1 1000 56 2 1 59 2 2 46 ... 2 1000 64 3 1 54 ... 100 1 451 100 2 153 ... 100 1000 21 I want to calculate moving average with window 10 for the value column. I guess I have to do something like df.groupby('object').apply(lambda ~calculate MA~) and then merge this Series to the original dataframe by object? Can't figure out exact commands 回答1: You can use rolling with transform : df['moving'] = df

Group by DataFrame with list and sum

落花浮王杯 提交于 2020-03-16 11:33:40
问题 I have a pandas Dataframe df and I want to Group by text column with aggregation of: Stack the english_word and return the list Sum the count column Now I only can do either making the english_word list or sum the count column. I try to do that, but it return error. How to do both of that aggregation? In simple, what I want: text saya eat chicken english_word [eat,chicken] count 2 df.groupby('text', as_index=False).agg({'count' : lambda x: x.sum(), 'english_word' : lambda x: x.list()}) This

Group by DataFrame with list and sum

喜欢而已 提交于 2020-03-16 11:33:38
问题 I have a pandas Dataframe df and I want to Group by text column with aggregation of: Stack the english_word and return the list Sum the count column Now I only can do either making the english_word list or sum the count column. I try to do that, but it return error. How to do both of that aggregation? In simple, what I want: text saya eat chicken english_word [eat,chicken] count 2 df.groupby('text', as_index=False).agg({'count' : lambda x: x.sum(), 'english_word' : lambda x: x.list()}) This

Group by DataFrame with list and sum

拥有回忆 提交于 2020-03-16 11:32:07
问题 I have a pandas Dataframe df and I want to Group by text column with aggregation of: Stack the english_word and return the list Sum the count column Now I only can do either making the english_word list or sum the count column. I try to do that, but it return error. How to do both of that aggregation? In simple, what I want: text saya eat chicken english_word [eat,chicken] count 2 df.groupby('text', as_index=False).agg({'count' : lambda x: x.sum(), 'english_word' : lambda x: x.list()}) This

Use pandas.shift() within a group

↘锁芯ラ 提交于 2020-03-15 07:28:28
问题 I have a dataframe with panel data, let's say it's time series for 100 different objects: object period value 1 1 24 1 2 67 ... 1 1000 56 2 1 59 2 2 46 ... 2 1000 64 3 1 54 ... 100 1 451 100 2 153 ... 100 1000 21 I want to add a new column prev_value that will store previous value for each object: object period value prev_value 1 1 24 nan 1 2 67 24 ... 1 99 445 1243 1 1000 56 445 2 1 59 nan 2 2 46 59 ... 2 1000 64 784 3 1 54 nan ... 100 1 451 nan 100 2 153 451 ... 100 1000 21 1121 Can I use

How to apply an accumulative custom aggregation function with a group by on Pandas

有些话、适合烂在心里 提交于 2020-03-04 18:06:13
问题 I have the following DataFrame df = pd.DataFrame({'model': ['A0', 'A0', 'A1', 'A1','A0', 'A0', 'A1', 'A1', 'A0', 'A0', 'A1', 'A1'], 'y_true': [1, 2, 3, 3, 4, 5, 6, 7, 8, 9, 10, 11], 'y_pred': [0, 1, 5, 5, 7, 8, 8, 12, 8, 7, 14, 15], 'week': [1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3]}, ) model y_true y_pred week 0 A0 1 0 1 1 A0 2 1 1 2 A1 3 5 1 3 A1 3 5 1 4 A0 4 7 2 5 A0 5 8 2 6 A1 6 8 2 7 A1 7 12 2 8 A0 8 8 3 9 A0 9 7 3 10 A1 10 14 3 11 A1 11 15 3 And I want to make some metrics calculus with

Understanding the execution of DataFrame in python

狂风中的少年 提交于 2020-03-03 07:08:59
问题 I am new to python and i want to understand how the execution takes place in a DataFrame. let's try this with an example from the dataset found in the kaggle.com( Titanic: Machine Learning from Disaster ). I wanted to replace the NaN value with the mean() for the respective sex . ie. the NaN value for Men should be replaced by the mean of the mens age and vice versa. now i achieved this by using this line of code _data['new_age']=_data['new_age'].fillna(_data.groupby('Sex')['Age'].transform(

Understanding the execution of DataFrame in python

£可爱£侵袭症+ 提交于 2020-03-03 07:07:34
问题 I am new to python and i want to understand how the execution takes place in a DataFrame. let's try this with an example from the dataset found in the kaggle.com( Titanic: Machine Learning from Disaster ). I wanted to replace the NaN value with the mean() for the respective sex . ie. the NaN value for Men should be replaced by the mean of the mens age and vice versa. now i achieved this by using this line of code _data['new_age']=_data['new_age'].fillna(_data.groupby('Sex')['Age'].transform(

Understanding the execution of DataFrame in python

喜你入骨 提交于 2020-03-03 07:07:21
问题 I am new to python and i want to understand how the execution takes place in a DataFrame. let's try this with an example from the dataset found in the kaggle.com( Titanic: Machine Learning from Disaster ). I wanted to replace the NaN value with the mean() for the respective sex . ie. the NaN value for Men should be replaced by the mean of the mens age and vice versa. now i achieved this by using this line of code _data['new_age']=_data['new_age'].fillna(_data.groupby('Sex')['Age'].transform(

Understanding the execution of DataFrame in python

拜拜、爱过 提交于 2020-03-03 07:07:11
问题 I am new to python and i want to understand how the execution takes place in a DataFrame. let's try this with an example from the dataset found in the kaggle.com( Titanic: Machine Learning from Disaster ). I wanted to replace the NaN value with the mean() for the respective sex . ie. the NaN value for Men should be replaced by the mean of the mens age and vice versa. now i achieved this by using this line of code _data['new_age']=_data['new_age'].fillna(_data.groupby('Sex')['Age'].transform(