Extract date from a string column containing timestamp in Pyspark

Question

I have a dataframe which has a date in the following format:

+----------------------+
|date                  |
+----------------------+
|May 6, 2016 5:59:34 AM|
         


        

Answer 1

There are some errors with your pattern. Here's a suggestion:

from_pattern = 'MMM d, yyyy h:mm:ss aa'
to_pattern = 'yyyy-MM-dd'
df.withColumn('part_date', from_unixtime(unix_timestamp(df['date'], from_pattern), to_pattern)).show()

+----------------------+----------+
|date                  |part_date |
+----------------------+----------+
|May 6, 2016 5:59:34 AM|2016-05-06|
+----------------------+----------+