spark submit throws an error java.lang.ClassNotFoundException: scala.runtime.java8.JFunction2$mcIII$sp

Question

I wrote a code for word count but when I tried running it from CMD in windows using below command it throws an exception.

spark-submit --class com.sample.WordCou         


        

Answer 1

The other answers are correct.

To add to them, don't forget to update the jar path in the spark-submit command when you change Scala versions. So if you're using sbt it's:

1. Update the Scala version to be compatible with Spark.
2. sbt package
3. Update the jar path in the spark-submit command. In my case it went from target/scala-2.12/word-count-app_2.12-1.0.jar to target/scala-2.11/word-count-app_2.11-1.0.jar.