Apache Spark running spark-shell on YARN error

Question

I downloaded: spark-2.1.0-bin-hadoop2.7.tgz from http://spark.apache.org/downloads.html. I have Hadoop HDFS and YARN started with $ start-dfs.sh and

Answer 1

I found the solution from another Stackoverflow question. It was not about configuring Apache Spark, it was about configuring Hadoop YARN:

Running yarn with spark not working with Java 8

Make sure your yarn-site.xml, from your Hadoop configuration folder, has these properties:

    yarn.nodemanager.pmem-check-enabled
    false



    yarn.nodemanager.vmem-check-enabled
    false