how to convert json string to dataframe on spark

Question

I want to convert string variable below to dataframe on spark.

val jsonStr = \"{ \"metadata\": { \"key\": 84896, \"value\": 54 }}\"

I know

Answer 1

There will be some error in some case like Illegal Patter component : XXX so for that you need to add .option with timestamp in spark.read so updated code will be.

val spark = SparkSession
          .builder()
          .master("local")
          .appName("Test")
          .getOrCreate()
import spark.implicits._
val jsonStr = """{ "metadata": { "key": 84896, "value": 54 }}"""
val df = spark.read.option("timestampFormat", "yyyy/MM/dd HH:mm:ss ZZ").json(Seq(jsonStr).toDS)
df.show()