I would like to set spark.eventLog.enabled and spark.eventLog.dir at the spark-submit or start-all level -- not require it to
spark.eventLog.enabled
spark.eventLog.dir
spark-submit
start-all
Create a local directory:
$ mkdir /tmp/spark-events
Run Spark-shell with --conf spark.eventLog.enabled
$ spark-shell --conf spark.eventLog.enabled --class com.MainClass --packages packages_if_any --master local[4] app.jar