Set your merge columns as index, and use df1.join(df2) instead https://stackoverflow.com/questions/40860457/improve-pandas-merge-performance