Use combineByKey to get output as (key, iterable[values])

Question

I am trying to transform RDD(key,value) to RDD(key,iterable[value]), same as output returned by the groupByKey method. But as

Answer 1

If you want a output similar from what groupByKey will give you, then you should absolutely use groupByKey and not some other method. The reduceByKey, combineByKey, etc. are only more efficient compared to using groupByKey followed with an aggregation (giving you the same result as one of the other groupBy methods could have given).

As the wanted result is an RDD[key,iterable[value]], building the list yourself or letting groupByKey do it will result in the same amount of work. There is no need to reimplement groupByKey yourself. The problem with groupByKey is not its implementation but lies in the distributed architecture.

For more information regarding groupByKey and these types of optimizations, I would recommend reading more here.