drop duplicates pandas dataframe

问题

I am getting an error message when using drop_duplicates to drop duplicate columns from my dataframe.

ValueError: Buffer has wrong number of dimensions (expected 1, got 2)

Below is a min example (notice that I don't have duplicate column names here, since that column won't be added then, so I var1 would be called var0 in my actual dataframe)

dict1 = [{'var0': 0, 'var1': 0, 'var2': 2},
         {'var0': 0, 'var1': 0, 'var2': 4},
         {'var0': 0, 'var1': 0, 'var2': 8},
         {'var0':0, 'var1': 0, 'var2': 12},]
df = pd.DataFrame(dict1, index=['s1', 's2','s1','s2'])
df.T.drop_duplicates().T

回答1:

The problem is with your indexing, when you transpose your DataFrame you will get duplicate column names which are messing it up. See below

dict1 = [{'var0': 0, 'var1': 0, 'var2': 2},
         {'var0': 0, 'var1': 0, 'var2': 4},
         {'var0': 0, 'var1': 0, 'var2': 8},
         {'var0':0, 'var1': 0, 'var2': 12},]
df = pd.DataFrame(dict1, index=['s1', 's2','s1','s2'])
df.reset_index().T.drop_duplicates().T.set_index('index')

来源：https://stackoverflow.com/questions/51470071/drop-duplicates-pandas-dataframe