YARN Resourcemanager not connecting to nodemanager

Question

thanks in advance for any help

I am running the following versions:

Hadoop 2.2 zookeeper 3.4.5 Hbase 0.96 Hive 0.12

When I go to http://:50070 I am a

Answer 1

I experienced an issue with very similar symptoms although it was the nodemanager not connecting to the resource manager. The problem was that in yarn-site.xml there is (or may be) a property named "yarn.nodemanager.hostname". That setting had been accidentally populated with the hostname of the HDFS "namenode" but it is supposed to contain the hostname of the YARN per-node "nodemanager". Depending on what was entered for other properties this was causing various errors like "Retrying connect", "Connection refused", or resource allocation errors. Setting this to "0.0.0.0" (the default) fixed the problem.

Answer 2

1. check YARN HA is enabled or not?
2. In case it is enabled then for each resource managers mentioned in the yarn-site.xml yarn.resourcemanager.ha.rm-ids (e.g. rm1,rm2) run the resourcemanager service.

Answer 3

You'll need to specify

hostname:port

Yes, it has to be set on all the nodes, not just the ResourceManager node (You'll be fine just copying the same hadoop's conf dir to all of your nodes). If you need some minimal working configs to get started, take a look here: toster.ru/q/57046#answer_208326

Answer 4

I had a very similar problem, and it was solved just by specifying the ResourceManager hostname, no need to spell out the exact address per service.

<property>
  <name>yarn.resourcemanager.hostname</name>
  <value>master-1</value>
</property>

Answer 5

The rsync or scp command can be used to copy the configuration files from the master node to the slave nodes:

for host in $HADOOP_CONF_DIR/slaves; do 
    rsync -rv $HADOOP_CONF_DIR/* $host:$HADOOP_CONF_DIR/
done

Note, here I am assuming all the nodes have the same hadoop directory layout.

Answer 6

I also had the same issue but in my case only one node manage was listed in resource manager. I placed below property in yarn-site.xml and I could see nodes listed at RM.

<property>
    <name>yarn.resourcemanager.hostname</name>
    <value><master-1></value>
</property>