Using reduceByKey in Apache Spark (Scala)

前端 未结 3 1658
囚心锁ツ
囚心锁ツ 2020-12-24 02:23

I have a list of Tuples of type : (user id, name, count).

For example,

val x = sc.parallelize(List(
    (\"a\", \"b\", 1),
    (\"a\", \"b\", 1),
           


        
3条回答
  •  离开以前
    2020-12-24 03:11

    The syntax is below:

    reduceByKey(func: Function2[V, V, V]): JavaPairRDD[K, V],
    

    which says for the same key in an RDD it takes the values (which will be definitely of same type) performs the operation provided as part of function and returns the value of same type as of parent RDD.

提交回复
热议问题