How to find which columns contain any NaN value in Pandas dataframe

前端 未结 8 1719
醉酒成梦
醉酒成梦 2020-11-28 02:17

Given a pandas dataframe containing possible NaN values scattered here and there:

Question: How do I determine which columns contain NaN values? In

8条回答
  •  感情败类
    2020-11-28 02:42

    In datasets having large number of columns its even better to see how many columns contain null values and how many don't.

    print("No. of columns containing null values")
    print(len(df.columns[df.isna().any()]))
    
    print("No. of columns not containing null values")
    print(len(df.columns[df.notna().all()]))
    
    print("Total no. of columns in the dataframe")
    print(len(df.columns))
    

    For example in my dataframe it contained 82 columns, of which 19 contained at least one null value.

    Further you can also automatically remove cols and rows depending on which has more null values
    Here is the code which does this intelligently:

    df = df.drop(df.columns[df.isna().sum()>len(df.columns)],axis = 1)
    df = df.dropna(axis = 0).reset_index(drop=True)
    

    Note: Above code removes all of your null values. If you want null values, process them before.

提交回复
热议问题