How to convert custom datetime format to timestamp?

Question

Any Idea why I am getting the result below?

scala> val b = to_timestamp($\"DATETIME\", \"ddMMMYYYY:HH:mm:ss\")
b: org.apache.spark.sql.Column = to_timesta         


        

Answer 1

try below code

I have created a sample dataframe "df" for the table

+---+-------------------+
| id|               date|
+---+-------------------+
|  1| 01JAN2017:01:02:03|
|  2| 15MAR2017:01:02:03|
|  3|02APR2015:23:24:25 |
+---+-------------------+
val t_s= unix_timestamp($"date","ddMMMyyyy:HH:mm:ss").cast("timestamp")

df.withColumn("ts",t_s).show()

+---+-------------------+--------------------+
| id|               date|                  ts|
+---+-------------------+--------------------+
|  1| 01JAN2017:01:02:03|2017-01-01 01:02:...|
|  2| 15MAR2017:01:02:03|2017-03-15 01:02:...|
|  3|02APR2015:23:24:25 |2015-04-02 23:24:...|
+---+-------------------+--------------------+

Thanks