Spark enable security with secure YARN Hadoop cluster
问题 I have an Hadoop 3.0 cluster configured with Kerberos. Everything works fine and YARN is started as well. Now I wish to add Spark on top of it and make full use of Hadoop and security. To do so I use a binary distribution of Spark 2.3 and modified the following. In spark-env.sh : YARN_CONF_DIR , set to the folder where my Hadoop configuration files core-site.xml , hdfs-site.xml and yarn-site.xml are located. In spark-defaults.conf : spark.master yarn spark.submit.deployMode cluster spark