Using log4j2 in Spark java application
I'm trying to use log4j 2 logger in my Spark job. Essential requirement: log4j2 config is located outside classpath, so I need to specify its location explicitly. When I run my code directly within IDE without using spark-submit , log4j2 works well. However when I submit the same code to Spark cluster using spark-submit , it fails to find log42 configuration and falls back to default old log4j. Launcher command ${SPARK_HOME}/bin/spark-submit \ --class my.app.JobDriver \ --verbose \ --master 'local[*]' \ --files "log4j2.xml" \ --conf spark.executor.extraJavaOptions="-Dlog4j.configurationFile