Mapping json to case class with Spark (spaces in the field name)

Question

I am trying to read a json file with the spark Dataset API, the problem is that this json contains spaces in some of the field names.

This would be a js

Answer 1

A workaround is to create a column name without space (give underscore) & rename the DF column to match the case class column name.

case class MyType(Field_Name: String)

dataframe.withColumnRenamed("Field Name", "Field_Name").as[MyType]

Answer 2

FYI I have created a Spark bug about this issue and they are fixing it https://issues.apache.org/jira/browse/SPARK-22442