Error while exploding a struct column in Spark

Question

I have a dataframe whose schema looks like this:

event: struct (nullable = true)
|    | event_category: string (nullable = true)
|    | event_name: string (n         


        

Answer 1

You may use following to flatten the struct. Explode does not work for struct as error message states.

val explodeDF = parquetDF.explode($"event") { 
case Row(properties: Seq[Row]) => properties.map{ property =>
  val errorCode = property(0).asInstanceOf[String]
  val errorDescription = property(1).asInstanceOf[String]
  Event(errorCode, errorDescription, email, salary)
 }
}.cache()
display(explodeDF)