Spark-Java:How to convert Dataset string column of format “yyyy-MM-ddThh:mm:ss.SSS+0000” to timestamp with a format?

岁酱吖の 提交于 2019-12-13 04:17:13

问题


I have a Dataset with one column lastModified of type string with format "yyyy-MM-ddThh:mm:ss.SSS+0000" (sample data: 2018-08-17T19:58:46.000+0000).

I have to add a new column lastModif_mapped of type Timestamp by converting the lastModified's value to format "yyyy-MM-dd hh:mm:ss.SSS".

I tried the code below, but the new column is getting the value null in it:

Dataset<Row> filtered = null;
filtered = ds1.select(ds1.col("id"),ds1.col("lastmodified"))
                .withColumn("lastModif_mapped", functions.unix_timestamp(ds1.col("lastmodified"), "yyyy-MM-dd HH:mm:ss.SSS").cast("timestamp")).alias("lastModif_mapped");

Where am I going wrong?

来源:https://stackoverflow.com/questions/54690305/spark-javahow-to-convert-dataset-string-column-of-format-yyyy-mm-ddthhmmss-s

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!