How to make Hadoop use all the cores on my system?

Question

I have a 32 core system. When I run a MapReduce job using Hadoop I never see the java process use more than 150% CPU (according to top) and it usually stays around the 100%

Answer 1

I think you need to set "mapreduce.framework.name" to "yarn",because the default value is "local".

put the following into your mapred-site.xml

mapreduce.framework.name
yarn