How to parse a csv that uses ^A (i.e. \001) as the delimiter with spark-csv?

Question

Terribly new to spark and hive and big data and scala and all. I\'m trying to write a simple function that takes an sqlContext, loads a csv file from s3 and returns a DataFr

Answer 1

With Spark 2.x and the CSV API, use the sep option:

val df = spark.read
  .option("sep", "\u0001")
  .csv("path_to_csv_files")