In spark, what does the parameter “minPartitions” works in SparkContext.textFile(path, minPartitions)?

Question

In Spark, either SparkContext or JavaSparkContext, there is one parameter which is minPartitions when you call sc.textFile. what does this parameter imply?

Answer 1

minPartitions will be passed to Hadoop's InputFormat.getSplits. The parameter is a hint, so you may get more or less partitions, depending on the Hadoop InputFormat implementation.