Sum values of PairRDD

后端 未结 2 2068
伪装坚强ぢ
伪装坚强ぢ 2021-01-01 05:56

I have an RDD of type:

dataset :org.apache.spark.rdd.RDD[(String, Double)] = MapPartitionRDD[26]

Which is equivalent to (Pedro, 0.083

2条回答
  •  执念已碎
    2021-01-01 06:02

    like this?:

    map(_._2).reduce((x, y) => x + y)
    

    breakdown: map the tuple to just the double values, then reduce the RDD by summing.

提交回复
热议问题