Why does reading csv file with empty values lead to IndexOutOfBoundException?

Question

I have a csv file with the foll struct

Name | Val1 | Val2 | Val3 | Val4 | Val5
John     1      2
Joe      1      2
David    1      2            10    11
         


        

Answer 1

You can do it as follows.

val df = sqlContext
         .read
         .textfile(csvFilePath)
         .map(_.split(delimiter_of_file, -1)
         .map(
             p => 
              Row(
                p(0), 
                p(1),
                p(2),
                p(3),
                p(4),
                p(5),
                p(6))

Split using delimiter of your file. When you set -1 as limit it consider all the empty fields.