Spark - How can get the Logical / Physical Query execution using - Thirft - Hive Interactor

Question

Spark - How can get the Logical / Physical Query execution using the following

1. Via Thrift
2. Via SparkInteractor

Answer 1

If you are using Spark 1, You can get the explain on a query this way:

sqlContext.sql("your SQL query").explain(true)

If you are using Spark 2, it's the same:

spark.sql("your SQL query").explain(true)

The same logic is available on a dataframe:

yourDF.explain(true) or yourDF.someOperation.explain(true)

Where someOperation could be maybe a select on a join or something else.