ClassNotFoundException scala.runtime.LambdaDeserialize when spark-submit

前端 未结 4 1610
南笙
南笙 2020-12-19 00:07

I follow the Scala tutorial on https://spark.apache.org/docs/2.1.0/quick-start.html

My scala file

/* SimpleApp.scala */
import org.apache.spark.Spark         


        
4条回答
  •  天命终不由人
    2020-12-19 00:50

    as @Alexey said, change Scala version to 2.11 fixed the problem.

    build.sbt

    name := "Simple Project"
    
    version := "1.0"
    
    scalaVersion := "2.11.11"
    
    libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.2.0" 
    

    Note that Scala version MUST MATCH with Spark. Look at the artifactId, spark-core_2.11 mean it was compatible with scala 2.11 (No backward or forward compatible)

提交回复
热议问题