Error while exploding a struct column in Spark

Question

I have a dataframe whose schema looks like this:

event: struct (nullable = true)
|    | event_category: string (nullable = true)
|    | event_name: string (n         


        

Answer 1

as the error message says, you can only explode array or map types, not struct type columns.

You can just do

df_json.withColumn("event_properties", $"event.properties")

This will generate a new column event_properties, which is also of struct-type

If you want to convert every element of the struct to a new column, then you cannot use withColumn, you need to do a select with a wildcard *:

df_json.select($"event.properties.*")