Spark Submit fails with java.lang.NoSuchMethodError: scala.Predef$.$conforms()Lscala/Predef$$less$colon$less;

Question

I am using spark 1.3.1 prebuild version spark-1.3.1-bin-hadoop2.6.tgz

Exception in thread \"main\" java.lang.NoSuchMethodError: scala.Predef$.$confo

Answer 1

The problem you are experiencing is due to the incompatibilities in Scala versions. Prebuild Spark 1.3.1 distribution is compiled with older Scala 2.10 because some of the Spark dependencies are not supported under 2.11, including JDBC support.

I would suggest to run your Spark cluster with Scala 2.10. However, if you want you can also compile your Spark package with Scala 2.11 in the following way:

dev/change-version-to-2.11.sh
mvn -Pyarn -Phadoop-2.4 -Dscala-2.11 -DskipTests clean package

Answer 2

i was experiencing same issue in scala IDE.and below steps resolved that.

Note:-check compatibility as per your scala-spark. for me it is scala version - 2.11.* is compatible with spark 2.4.*

Go the project >> right click >> properties >> scala compiler >> select "use project settings" option >> and change "scala installation" >> apply >> apply and close..... good to go.

click on below image link to see setting of Scala IDE