difference between rdd.collect().toMap to rdd.collectAsMap()?

Question

Is there any performance impact when I use collectAsMap on my RDD instead of rdd.collect().toMap ?

I have a key value rdd and I want to convert to HashMap as far I

Answer 1

The implementation of collectAsMap is the following

def collectAsMap(): Map[K, V] = self.withScope {
    val data = self.collect()
    val map = new mutable.HashMap[K, V]
    map.sizeHint(data.length)
    data.foreach { pair => map.put(pair._1, pair._2) }
    map
  }

Thus, there is no performance difference between collect and collectAsMap, because collectAsMap calls under the hood also collect.

Answer 2

No difference. Avoid using collect() as much as you can as it destroys the concept of parallelism and collects the data on the driver.