Apache spark: setting spark.eventLog.enabled and spark.eventLog.dir at submit or Spark start

前端 未结 2 1399
遇见更好的自我
遇见更好的自我 2021-02-01 07:47

I would like to set spark.eventLog.enabled and spark.eventLog.dir at the spark-submit or start-all level -- not require it to

2条回答
  •  萌比男神i
    2021-02-01 08:27

    Create a local directory:

    $ mkdir /tmp/spark-events
    

    Run Spark-shell with --conf spark.eventLog.enabled

    $ spark-shell --conf spark.eventLog.enabled --class com.MainClass --packages packages_if_any --master local[4] app.jar
    

提交回复
热议问题