pandas-groupby

I had a question earlier which is deleted and now modified to a less verbose form for you to read easily. I have a dataframe as given below df = pd.DataFrame({'subject_id' :[1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2],'day':[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20] , 'PEEP' :[7,5,10,10,11,11,14,14,17,17,21,21,23,23,25,25,22,20,26,26,5,7,8,8,9,9,13,13,15,15,12,12,15,15,19,19,19,22,22,15]}) df['fake_flag'] = '' I would like to fill values in column fake_flag based on the below rules 1) if preceding

Edit: the rookie mistake I made in string np.nan having pointed out by @coldspeed, @wen-ben, @ALollz. Answers are quite good, so I don't delete this question to keep those answers. Original: I have read this question/answer What's the difference between groupby.first() and groupby.head(1)? That answer explained that the differences are on handling NaN value. However, when I call groupby with as_index=False , they both pick NaN fine. Furthermore, Pandas has groupby.nth with similar functionality to head , and first What are difference of groupby.first(), groupby.nth(0), groupby.head(1) with as

I have given two dataframes below for you to test df = pd.DataFrame({ 'subject_id':[1,1,1,1,1,1,1,1,1,1,1], 'time_1' :['2173-04-03 12:35:00','2173-04-03 17:00:00','2173-04-03 20:00:00','2173-04-04 11:00:00','2173-04-04 11:30:00','2173-04-04 12:00:00','2173-04-05 16:00:00','2173-04-05 22:00:00','2173-04-06 04:00:00','2173-04-06 04:30:00','2173-04-06 06:30:00'], 'val' :[5,5,5,10,5,10,5,8,3,8,10] }) df1 = pd.DataFrame({ 'subject_id':[1,1,1,1,1,1,1,1,1,1,1], 'time_1' :['2173-04-03 12:35:00','2173-04-03 12:50:00','2173-04-03 12:59:00','2173-04-03 13:14:00','2173-04-03 13:37:00','2173-04-04 11:30:00

i have a csv file containing 4 years of data and i am trying to group data per season over the 4 years , differently saying, i need to summarize and plot my whole data into 4 season only . here's a look on my data file : timestamp,heure,lat,lon,impact,type 2006-01-01 00:00:00,13:58:43,33.837,-9.205,10.3,1 2006-01-02 00:00:00,00:07:28,34.5293,-10.2384,17.7,1 2007-02-01 00:00:00,23:01:03,35.0617,-1.435,-17.1,2 2007-02-02 00:00:00,01:14:29,36.5685,0.9043,36.8,1 2008-01-01 00:00:00,05:03:51,34.1919,-12.5061,-48.9,1 2008-01-02 00:00:00,05:03:51,34.1919,-12.5061,-48.9,1 .... 2011-12-31 00:00:00,05

I have to following df: Col1 Col2 test Something test2 Something test3 Something test Something test2 Something test5 Something I want to get Col1 Col2 Occur test Something 2 test2 Something 2 test3 Something 1 test Something 2 test2 Something 2 test5 Something 1 I've tried to use: df["Occur"] = df["Col1"].value_counts() But it didn't help. I've got Occur column full of 'NaN' groupby on 'col1' and then apply transform on Col2 to return a Series with its index aligned to the original df so you can add it as a column: In [3]: df['Occur'] = df.groupby('Col1')['Col2'].transform(pd.Series.value

I'm posting this because the topic just got brought up in another question/answer and the behavior isn't very well documented. Consider the dataframe df df = pd.DataFrame(dict( A=list('xxxyyy'), B=[np.nan, 1, 2, 3, 4, np.nan] )) A B 0 x NaN 1 x 1.0 2 x 2.0 3 y 3.0 4 y 4.0 5 y NaN I wanted to get the first and last rows of each group defined by column 'A' . I tried df.groupby('A').B.agg(['first', 'last']) first last A x 1.0 2.0 y 3.0 4.0 However, This doesn't give me the np.NaN s that I expected. How do I get the actual first and last values in each group? One option is to use the .nth method: