Parse a Pandas column to Datetime when importing table from SQL database and filtering rows by date

Question

I have a DataFrame with column named date. How can we convert/parse the \'date\' column to a DateTime object?

I loaded the dat

Answer 1

Don't confuse datetime.date with Pandas pd.Timestamp

A "Pandas datetime series" contains pd.Timestamp elements, not datetime.date elements. The recommended solution for Pandas:

s = pd.to_datetime(s)    # convert series to Pandas
mask = s > '2018-03-10'  # calculate Boolean mask against Pandas-compatible object

The top answers have issues:

• @RyanSaxe's accepted answer's first attempt doesn't work; the second answer is inefficient.
• As of Pandas v0.23.0, @Keith's highly upvoted answer doesn't work; it gives TypeError.

Any good Pandas solution must ensure:

1. The series is a Pandas datetime series, not object dtype.
2. The datetime series is compared to a compatible object, e.g. pd.Timestamp, or string in the correct format.

Here's a demo with benchmarking, demonstrating that the one-off cost of conversion can be immediately offset by a single operation:

from datetime import date

L = [date(2018, 1, 10), date(2018, 5, 20), date(2018, 10, 30), date(2018, 11, 11)]
s = pd.Series(L*10**5)

a = s > date(2018, 3, 10)             # accepted solution #2, inefficient
b = pd.to_datetime(s) > '2018-03-10'  # more efficient, including datetime conversion

assert a.equals(b)                    # check solutions give same result

%timeit s > date(2018, 3, 10)                  # 40.5 ms
%timeit pd.to_datetime(s) > '2018-03-10'       # 33.7 ms

s = pd.to_datetime(s)

%timeit s > '2018-03-10'                       # 2.85 ms