Slow loop python to search data in antoher data frame in python

Question

I have two data frames : one with all my data (called \'data\') and one with latitudes and longitudes of different stations where each observation starts and ends (called \'

Answer 1

This is one solution. You can also use pandas.merge to add 2 new columns to data and perform the equivalent mapping.

# create series mappings from info
s_lat = info.set_index('station')['latitude']
s_lon = info.set_index('station')['latitude']

# calculate Boolean mask on year
mask = data['year'] == '2018'

# apply mappings, if no map found use fillna to retrieve original data
data.loc[mask, 'latitude'] = data.loc[mask, 'station'].map(s_lat)\
                                 .fillna(data.loc[mask, 'latitude'])

data.loc[mask, 'longitude'] = data.loc[mask, 'station'].map(s_lon)\
                                  .fillna(data.loc[mask, 'longitude'])