NoClassDefFoundError com.apache.hadoop.fs.FSDataInputStream when execute spark-shell

Question

I\'ve downloaded the prebuild version of spark 1.4.0 without hadoop (with user-provided Haddop). When I ran the spark-shell command, I got this error:

> E         


        

Answer 1

for my case

running spark job locally differs from running it on cluster. on cluster you might have a different dependency/context to follow. so essentially in your pom.xml you might have dependencies declared as provided.

when running locally, you don't need these provided dependencies. just uncomment them and rebuild again.