Why does spark-submit and spark-shell fail with “Failed to find Spark assembly JAR. You need to build Spark before running this program.”?

Question

I was trying to run spark-submit and I get \"Failed to find Spark assembly JAR. You need to build Spark before running this program.\" When I try to run spark-shell

Answer 1

Your Spark package doesn't include compiled Spark code. That's why you got the error message from these scripts spark-submit and spark-shell.

You have to download one of pre-built version in "Choose a package type" section from the Spark download page.