How to check the number of partitions of a Spark DataFrame without incurring the cost of .rdd

Question

There are a number of questions about how to obtain the number of partitions of a n RDD and or a DataFrame : the answers invariably are:

Answer 1

In my experience df.rdd.getNumPartitions is very fast, I never encountered taking this more than a second or so.

Alternatively, you could also try

val numPartitions: Long = df
      .select(org.apache.spark.sql.functions.spark_partition_id()).distinct().count()

which would avoid using .rdd