Python: How to drop a row whose particular column is empty/NaN?

Question

I have a csv file. I read it:

import pandas as pd
data = pd.read_csv(\'my_data.csv\', sep=\',\')
data.head()

It has output like:

Answer 1

You can use the method dropna for this:

data.dropna(axis=0, subset=('sms', ))

See the documentation for more details on the parameters.

Of course there are multiple ways to do this, and there are some slight performance differences. Unless performance is critical, I would prefer the use of dropna() as it is the most expressive.

import pandas as pd
import numpy as np

i = 10000000

# generate dataframe with a few columns
df = pd.DataFrame(dict(
    a_number=np.random.randint(0,1e6,size=i),
    with_nans=np.random.choice([np.nan, 'good', 'bad', 'ok'], size=i),
    letter=np.random.choice(list('abcdefghijklmnop'), size=i))
                 )

# using notebook %%timeit
a = df.dropna(subset=['with_nans'])
#1.29 s ± 112 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

# using notebook %%timeit
b = df[~df.with_nans.isnull()]
#890 ms ± 59.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

# using notebook %%timeit
c = df.query('with_nans == with_nans')
#1.71 s ± 100 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)