Spark worker can not connect to Master

Question

While starting the worker node I get the following error :

Spark Command: /usr/lib/jvm/default-java/bin/java -cp /home/ubuntu/spark-1.5.1-bin-hadoop2.6/sbin/         


        

Answer 1

So, after some tinkering around I found that slave was not able to communicate with Master on the given port. I changed the security access rules and enabled all TCP traffic on all ports . This solved the problem.

To check if the port is open :

telnet master.ip master.port

The default port is 7077.

My spark-env.sh :

export SPARK_WORKER_INSTANCES=2 export SPARK_MASTER_IP=