Application report for application_ (state: ACCEPTED) never ends for Spark Submit (with Spark 1.2.0 on YARN)

Question

I am running kinesis plus spark application https://spark.apache.org/docs/1.2.0/streaming-kinesis-integration.html

I am running as below

command on ec2 inst

Answer 1

I faced the same issue in clouera quickstart VM when I tried to execute pyspark shell . Whe i see the job logs in resourcemanager , i see

17/02/18 22:20:53 ERROR yarn.ApplicationMaster: Failed to connect to driver at RM IP. 

That means job is not able to connect to RM (resource manager) because by default pyspark try to launch in yarn mode in cloudera VM .

pyspark --master local 

worked for me . Even starting RM s resolved the issue.

Thanks