Retrieve SparkContext from SparkSession

后端 未结 2 1094
闹比i
闹比i 2021-02-18 23:09

I\'m running a spark batch job and uses SparkSession as I need a lot of spark-sql features to process in each of my components. The SparkContext is in

相关标签:
2条回答
  • 2021-02-18 23:50

    The sparkContext field does not seem to be public anymore(I am using Spark 2.3.2), however, you can retreive it using a method of the same name:

    spark.sparkContext()
    

    This is applicable to Spark Java only.

    0 讨论(0)
  • 2021-02-19 00:03

    Just to post as an answer - the SparkContext can be accessed from SparkSession using spark.sparkContext (no parenthesis)

    0 讨论(0)
提交回复
热议问题