I have an Spark app which runs with no problem in local mode,but have some problems when submitting to the Spark cluster.
The error msg are as follows:
Worked for me after replacing
SparkConf sparkConf = new SparkConf().setAppName("SOME APP NAME");
with
SparkConf sparkConf = new SparkConf().setAppName("SOME APP NAME").setMaster("local[2]").set("spark.executor.memory","1g");
Found this solution on some other thread on stackoverflow.