Dataframe from List in Java

Question

• Spark Version : 1.6.2
• Java Version: 7

I have a List data. Something like:

[[dev, engg, 10000], [kar         


        

Answer 1

Task can be completed without JSON, on Scala:

val data = List("dev, engg, 10000", "karthik, engg, 20000")
val intialRdd = sparkContext.parallelize(data)
val splittedRDD = intialRdd.map(current => {
  val array = current.split(",")
  (array(0), array(1), array(2))
})
import sqlContext.implicits._
val dataframe = splittedRDD.toDF("name", "degree", "salary")
dataframe.show()

Output is:

+-------+------+------+
|   name|degree|salary|
+-------+------+------+
|    dev|  engg| 10000|
|karthik|  engg| 20000|
+-------+------+------+

Note: (array(0), array(1), array(2)) is a Scala Tuple