Property spark.yarn.jars - how to deal with it?

Question

My knowledge with Spark is limited and you would sense it after reading this question. I have just one node and spark, hadoop and yarn are installed on it.

I was abl

Answer 1

I was finally able to make sense of this property. I found by hit-n-trial that correct syntax of this property is

spark.yarn.jars=hdfs://xx:9000/user/spark/share/lib/*.jar

I didn't put *.jar in the end and my path was just ended with /lib. I tried putting actual assembly jar like this - spark.yarn.jars=hdfs://sanjeevd.brickred:9000/user/spark/share/lib/spark-yarn_2.11-2.0.1.jar but no luck. All it said that unable to load ApplicationMaster.

I posted my response to a similar question asked by someone at https://stackoverflow.com/a/41179608/2332121