How to increase the mappers and reducers in hadoop according to number of instances used to increase the performance?

前端 未结 4 967
说谎
说谎 2020-12-18 17:11

If I increase the number of mappers and decrease the number of reducers, then is there any difference in the performance (increase/decrease) of any job while execution?

4条回答
  •  予麋鹿
    予麋鹿 (楼主)
    2020-12-18 17:57

    You can increase number of mappers based on the block size and split size. One of the easiest way is to decrease the split size as shown below:

    Configuration conf= new Cofiguration();
    //set the value that increases your number of splits.
    conf.set("mapred.max.split.size", "1020");
    Job job = new Job(conf, "My job name");
    

提交回复
热议问题