How to compute the inverse of a RowMatrix in Apache Spark?

Question

I have a X, distributed matrix, in RowMatrix form. I am using Spark 1.3.0. I need to be able to calculate X inverse.

Answer 1

Matrix U returned by X.computeSVD has dimensions m x k where m is the number of rows of the original (distributed) RowMatrix X. One would expect m to be large (possibly larger than k), so it is not advisable to collect it in the driver if we want our code to scale to really large values of m.

I would say both of the solutions below suffer from this flaw. The answer given by @Alexander Kharlamov calls val U = svd.U.toBlockMatrix().toLocalMatrix() which collects the matrix in the driver. The same happens with the answer given by @Climbs_lika_Spyder (btw your nick rocks!!), which calls svd.U.rows.collect.flatMap(x => x.toArray). I would rather suggest to rely on a distributed matrix multiplication such as the Scala code posted here.