Spark: Sort records in groups?

Question

I have a set of records which I need to:

1) Group by \'date\', \'city\' and \'kind\'

2) Sort every group by \'prize

In my code:

impor         


        

Answer 1

Replace map with flatMap

val x = rsGrp.map{r => 
  val lst = r.toList
  lst.map{e => (e.prize, e)}
  }

this will give you a

org.apache.spark.rdd.RDD[(Int, Record)] = FlatMappedRDD[10]

and then you can call sortBy(_._1) on the RDD above.