Reshift Nan values column dataframe if match with list

问题

I want to rearrange column name values that contains Nan.

Condition that i want is, if string in list match with column[1], it will only reshift column values that contain row under matched string, so its my dataframe before shifted.

[in] : df
[Out]:

   column1     column2    column3 
0  aba abab    800.0      900.0
1  aaa acc     900.0      60.0 
2  bba jka     809.0      400.0
3  fff yy      521.0      490.0  
4  hkm asa j   290.0      321.0    
5  daa rr oo   88.0       Nan
6  jtuy ww ddw Nan        600.0
8  bkam ftf    Nan        Nan   
9  fgqefc      Nan        Nan
10 daas we fg  Nan        Nan   
11 judv mm mk  Nan        Nan   
12 hus gg hhh  Nan        Nan

and here my list

my_list= ['bba jka', 'hkm asa j']

so it my dataframe that i wanted, which name is df1

column1     column2    column3 
0  aba abab    800.0      900.0
1  aaa acc     900.0      60.0 
2  bba jka     Nan        Nan
3  fff yy      809.0      400.0  
4  hkm asa j   Nan        Nan    
5  daa rr oo   521.0      490.0
6  jtuy ww ddw 290.0      321.0
8  bkam ftf    88.0       Nan   
9  fgqefc      Nan        600.0
10 daas we fg  Nan        Nan   
11 judv mm mk  Nan        Nan   
12 hus gg hhh  Nan        Nan

I dont understand how to achieve df1 with shift and match, anyone can solve it?

回答1:

Here's a suggestion, which might not be optimal:

Step 1: Preparations for apply:

match = df['column1'].str.fullmatch('|'.join(entry for entry in my_list))
df['shift'] = match.cumsum()
df['index'] = df.index
df.set_index('column1', drop=True, inplace=True)

Result (df) looks like:

            column2 column3  shift  index
column1                                  
aba abab      800.0   900.0      0      0
aaa acc       900.0    60.0      0      1
bba jka       809.0   400.0      1      2
fff yy        521.0   490.0      1      3
hkm asa j     290.0   321.0      2      4
daa rr oo      88.0     NaN      2      5
...

Step 2: "Shifting" via apply and NaN assingment via mask match:

df = df.apply(lambda row: df.shift(int(row.at['shift'])).iloc[int(row.at['index'])],
              axis='columns')
df[list(match)] = np.nan

Step 3: Clean up:

df.drop(['shift', 'index'], axis='columns', inplace=True)
df.reset_index(inplace=True)

The result is hopefully as expected:

        column1 column2 column3
0      aba abab   800.0   900.0
1       aaa acc   900.0    60.0
2       bba jka     NaN     NaN
3        fff yy   809.0   400.0
4     hkm asa j     NaN     NaN
5     daa rr oo   521.0   490.0
6   jtuy ww ddw   290.0   321.0
7      bkam ftf    88.0     NaN
8        fgqefc     NaN   600.0
9    daas we fg     NaN     NaN
10   judv mm mk     NaN     NaN
11   hus gg hhh     NaN     NaN

But I don't like the use of df.shift in apply. The problem is that a possible match in the first row would lead to a false result without shift. Here's a version that avoids this problem and is more straight forward in apply:

# Preparation
df = pd.concat(
        [pd.DataFrame({col: ['NOT IN LIST' if i == 0 else np.nan]
                       for i, col in enumerate(df.columns)}), df],
        axis='index', 
        ignore_index=True
    )
match = df['column1'].str.fullmatch('|'.join(entry for entry in my_list))
df['shift'] = df.index - match.cumsum()
df.set_index('column1', drop=True, inplace=True)

# Shifting etc.
df = df.apply(lambda row: df.iloc[int(row.at['shift'])], axis='columns')
df[list(match)] = np.nan

# Clean up
df.drop('NOT IN LIST', axis='index', inplace=True)
df.drop('shift', axis='columns', inplace=True)
df.reset_index(inplace=True)

(The assumption here is that the string 'NOT IN LIST' is not in my_list. Most likely the empty string '' would be a good choice too.)

来源：https://stackoverflow.com/questions/64762456/reshift-nan-values-column-dataframe-if-match-with-list

标签

python

pandas

list

dataframe

shift