Save Spark org.apache.spark.mllib.linalg.Matrix to a file

Question

The result of correlation in Spark MLLib is a of type org.apache.spark.mllib.linalg.Matrix. (see http://spark.apache.org/docs/1.2.1/mllib-statistics.html#correlations)

Answer 1

The answer by Dylan Hogg was great, to enhance it slightly, add a column index. (In my use case, once I created a file and downloaded it, it was not sorted due to the nature of parallel process etc.)

ref: https://www.safaribooksonline.com/library/view/scala-cookbook/9781449340292/ch10s12.html

substitute with this line and it will put a sequence number on the line (starting w/ 0) making it easier to sort when you go to view it

val lines: List[String] = localMatrix 
  .map(line => line.mkString(" ")) 
  .zipWithIndex.map { case(line, count) => s"$count $line" }