Convert Row to map in spark scala

Question

I have a row from a data frame and I want to convert it to a Map[String, Any] that maps column names to the values in the row for that column.

Is there an easy way to do

Answer 1

Let's say you have a data Frame with these columns:

[time(TimeStampType), col1(DoubleType), col2(DoubleType)]

You can do something like this:

val modifiedDf = df.map{row => 
    val doubleObject = row.getValuesMap(Seq("col1","col2"))
    val timeObject = Map("time" -> row.getAs[TimeStamp]("time"))
    val map = doubleObject ++ timeObject
}