Spark dataframe reducebykey like operation

Question

I have a Spark dataframe with the following data (I use spark-csv to load the data in):

key,value
1,10
2,12
3,0
1,20
         


        

Answer 1

I think user goks missed out on some part in the code. Its not a tested code.

.map should have been used to convert the rdd to a pairRDD using .map(lambda x: (x,1)).reduceByKey. ....

reduceByKey is not available on a single value rdd or regular rdd but pairRDD.

Thx