Pandas concat flips all my values in the DataFrame

问题

I have a dataframe called 'running_tally'

        list   jan_to  jan_from
0         LA    True      False
1         NY   False       True

I am trying to append new data to it in the form of a single column dataframe called 'new_data'

        list   
0        HOU
1         LA

I concat these two dfs based on their 'list' column for further processing, but immediately after I do that all the boolean values unexpectedly flip.

running_tally = pd.concat([running_tally,new_data]).groupby('list',as_index=False).first()

the above statement will produce:

        list   jan_to  jan_from
0         LA    False      True
1         NY     True     False
2        HOU     NaN        NaN

NaN values are expected for the new row, but I don't know why the bools all flip. What could be the reason for this? The code logically makes sense to me so I'm not sure where I'm going wrong. Thanks

EDIT: I made an edit to 'new_data' to include a repeat with LA. The final output should not have repeats which my code currently handles correctly, just has boolean flipping

EDIT 2: Turns out that when concatenating, the columns would flip in order leading me to believe the bools flipped. Still an open issue however

回答1:

I am not sure why you want to use a groupby in this case... when using concat there is no need to specify which columns you want to use, as long as their names are identical. Simple concatenation like this should do:

running_tally = pd.concat([running_tally,new_data], ignore_index=True, sort=False)

EDIT to take question edit into account: this should do the same job, without duplicates.

running_tally = running_tally.merge(new_data, on="list", how="outer")

回答2:

I don´t get the booleans flipped as you, but you can try this too:

running_tally=running_tally.append(new_data,ignore_index=True)
print(running_tally)

Output:

  list jan_to jan_from
0   LA   True    False
1   NY  False     True
2  HOU    NaN      NaN

EDIT: Since the question was edited, you could try with:

running_tally=running_tally.append(new_data,ignore_index=True).groupby('list',as_index=False).first()

回答3:

The actual row order was being flipped when using concat for pandas 0.20.1

How to concat pandas Dataframes without changing the column order in Pandas 0.20.1?

来源：https://stackoverflow.com/questions/62665453/pandas-concat-flips-all-my-values-in-the-dataframe