Overwrite columns in DataFrames of different sizes pandas

Question

I have following two Data Frames:

df1 = pd.DataFrame({\'ids\':[1,2,3,4,5],\'cost\':[0,0,1,1,0]})
df2 = pd.DataFrame({\'ids\':[1,5],\'cost\':[1,4]})
         


        

Answer 1

You could do this with a left merge:

merged = pd.merge(df1, df2, on='ids', how='left')
merged['cost'] = merged.cost_x.where(merged.cost_y.isnull(), merged['cost_y'])
result = merged[['ids','cost']]

However you can avoid the need for the merge (and get better performance) if you set the ids as an index column; then pandas can use this to align the results for you:

df1 = df1.set_index('ids')
df2 = df2.set_index('ids')

df1.cost.where(~df1.index.isin(df2.index), df2.cost)
ids
1    1.0
2    0.0
3    1.0
4    1.0
5    4.0
Name: cost, dtype: float64