MemoryError when I merge two Pandas data frames

Question

I searched almost all over the internet and somehow none of the approaches seem to work in my case.

I have two large csv files (each with a million+ rows and about

Answer 1

The reason you might be getting MemoryError: Unable to allocate.. could be due to duplicates or blanks in your dataframe. Check the column you are joining on (when using merge) and see if you have duplicates or blanks. If so get rid of them using this command:

df.drop_duplicates(subset ='column_name', keep = False, inplace = True) 

Then re-run your python/pandas code. This worked for me.