How to use Pandas to merge multiple big CSV files inside a folder?

Question

First of all, each file has multiple columns (more tan 1000) and a size of about 44Mb, some headers are repeated through the csv files and some are unique to certain file, t