Read csv as Data Frame in spark 1.6

Question

I have Spark 1.6 and trying to read a csv (or tsv) file as a dataframe. Here are the steps I take:

scala>  val sqlContext= new org.apache.spark.sql.SQLCon         


        

Answer 1

In java first add dependency in POM.xml file and run following code to read csv file.

            com.databricks
            spark-csv_2.10
            1.4.0
        

Dataset df = sparkSession.read().format("com.databricks.spark.csv").option`enter code here`("header", true).option("inferSchema", true).load("hdfs://localhost:9000/usr/local/hadoop_data/loan_100.csv");