问题
I am able to do a spark-submit to my cloudera cluster. the job dies after a few minutes with exceptions complaining it can not find various classes. These are classes that are in the spark dependency path. I keep adding the jars one at a time using command line args --jars, the yarn log keeps dumping out the next jar it can't find.
What setting allows the spark/yarn job to find all the dependent jars?
I already set the "spark.home" attribute to the correct path - /opt/cloudera/parcels/CDH/lib/spark
回答1:
I found it!
remove
.set("spark.driver.host", "driver computer ip address")
from your driver code.
来源：https://stackoverflow.com/questions/25495661/spark-submit-to-cloudera-cluster-can-not-find-any-dependent-jars