Submit Spark job is not loading spark-cloudant:2.0.0-s_2_11 package

问题

I am submitting Spark using below command,

./spark-submit --packages cloudant-labs:spark-cloudant:2.0.0-s_2.11 --class spark.cloudant.connecter.cloudantconnecter --master local[*] /opt/demo/sparkScripts/ScoredJob/sparkcloudantconnecter.jar

But it seems "spark-cloudant" package is not loading.. Might be it is not loading because of artifact located at Spark Packages repository(https://dl.bintray.com/spark-packages/maven/)

So What changes is require in command to load the package?

Full details of This package is showing here https://mvnrepository.com/artifact/cloudant-labs/spark-cloudant/2.0.0-s_2.11

回答1:

It seems require to add repository while loading package using Spark-submit command,

-- repository url

So new command will be,

./spark-submit --packages cloudant-labs:spark-cloudant:2.0.0-s_2.11 --repositories https://dl.bintray.com/spark-packages/maven/ --class spark.cloudant.connecter.cloudantconnecter --master local[*] /opt/demo/sparkScripts/ScoredJob/sparkcloudantconnecter.jar

来源：https://stackoverflow.com/questions/40927252/submit-spark-job-is-not-loading-spark-cloudant2-0-0-s-2-11-package