How to convert custom datetime format to timestamp?

Question

Any Idea why I am getting the result below?

scala> val b = to_timestamp($\"DATETIME\", \"ddMMMYYYY:HH:mm:ss\")
b: org.apache.spark.sql.Column = to_timesta         


        

Answer 1

Use y (year) not Y (week year):

spark.sql("SELECT to_timestamp('04JUN2018:10:11:12', 'ddMMMyyyy:HH:mm:ss')").show
// +--------------------------------------------------------+
// |to_timestamp('04JUN2018:10:11:12', 'ddMMMyyyy:HH:mm:ss')|
// +--------------------------------------------------------+
// |                                     2018-06-04 10:11:12|
// +--------------------------------------------------------+

Another example:

scala> sql("select to_timestamp('12/08/2020 1:24:21 AM', 'MM/dd/yyyy H:mm:ss a')").show
+-------------------------------------------------------------+
|to_timestamp('12/08/2020 1:24:21 AM', 'MM/dd/yyyy H:mm:ss a')|
+-------------------------------------------------------------+
|                                          2020-12-08 01:24:21|
+-------------------------------------------------------------+