I am surprised to find that adding a column to dataframe so time consuming, and after doing a reset_index can improve the performance to such a great deal. Below is the code