How does Hadoop decide how many nodes will perform the Map and Reduce tasks?

Question

I\'m new to hadoop and I\'m trying to understand it. Im talking about hadoop 2. When I have an input file which I wanto to do a MapReduce, in the MapReduce programm I say th

Answer 1

how does Hadoop decides how many nodes will do map tasks

By default, the number of mappers will be same as the number of split (blocks) of the input to the mapreduce.

Now about the nodes, In the Hadoop 2, each node runs it own NodeManager (NM). The job of the NM is to manage the application container assigned to it by the Resourcemanager (RM). So basically, each of the task will be running in the individual container. To run the mapper tasks, ApplicationMaster negotiate the container from the ResourceManager. Once the containers are allocated, the NodeManager will launch the task and monitor it.

which nodes will do the reduce tasks?

Again the reduce tasks will also runs in the containers. The ApplicationMaster (per-application (job)) will negotiate the containers from the RM and launch the reducer tasks. Mostly they run on the different nodes then the Mapper nodes.

The default number of reducers for any job is 1. The number of reducers can be set in the job configuration.