ClassNotFoundException scala.runtime.LambdaDeserialize when spark-submit

Question

I follow the Scala tutorial on https://spark.apache.org/docs/2.1.0/quick-start.html

My scala file

/* SimpleApp.scala */
import org.apache.spark.Spark         


        

Answer 1

I have similar issue while following the instructions provided at https://spark.apache.org/docs/2.4.3/quick-start.html

My setup details: Spark version: 2.4.3 Scala version: 2.12.8

However, when i changed my sbt file to below configuration everything worked fine.(both compilation and running the application jar)

name := "Simple Project"

version := "1.0"

scalaVersion := "2.11.11"

libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.4.3"

It looks like spark 2.4.3 is compatible with 2.11.11 Scala version only. While compiling the sample project sbt has downloaded the Scala 2.11 library from "https://repo1.maven.org/maven2/org/scala-lang/scala-library/2.11.11"