Reading TSV into Spark Dataframe with Scala API

后端 未结 2 1365
终归单人心
终归单人心 2020-12-03 10:02

I have been trying to get the databricks library for reading CSVs to work. I am trying to read a TSV created by hive into a spark data frame using the scala api.

Her

2条回答
  •  旧时难觅i
    2020-12-03 10:26

    All of the option parameters are passed in the option() function as below:

    val segments = sqlContext.read.format("com.databricks.spark.csv")
        .option("delimiter", "\t")
        .load("s3n://michaeldiscenza/data/test_segments")
    

提交回复
热议问题