MapReduce job hangs, waiting for AM container to be allocated

Question

I tried to run simple word count as MapReduce job. Everything works fine when run locally (all work done on Name Node). But, when I try to run it on a cluster using YARN (ad

Answer 1

Old question, but I got on the same issue recently and in my case it was due to manually setting the master to local in the code.

Please, search for conf.setMaster("local[*]") and remove it.

Hope it helps.

Answer 2

Check your hosts file on master and slave nodes. I had exactly this problem. My hosts file looked like this on master node for example

127.0.0.0 localhost
 127.0.1.1 master-virtualbox
 192.168.15.101 master 

I changed it like below

192.168.15.101 master master-virtualbox localhost

So it worked.

Answer 3

These lines

<property>
  <name>yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage</name>
  <value>100</value>
</property>

in the yarn-site.xml solved my problem since the node will be marked as unhealthy when disk usage is >=95%. Solution mainly suitable for pseudodistributed mode.