unsupportedOperationException Error converting string to DateTime using Joda time

Question

I am converting string to datetime field using joda.time.Datetime libraries but it throws unsupported exception Here is main class code:

//create new var with          


        

Answer 1

1. As you can read in the official documentation dates in Spark SQL are represented using java.sql.Timestamp. If you want to use Joda time you have to convert output to the correct type

2. SparkSQL can easily handle standard date formats using type casting:

   sc.parallelize(Seq(Tuple1("2016-01-11 00:01:02")))
     .toDF("dt")
     .select($"dt".cast("timestamp"))