How to increase the mappers and reducers in hadoop according to number of instances used to increase the performance?

前端 未结 4 968
说谎
说谎 2020-12-18 17:11

If I increase the number of mappers and decrease the number of reducers, then is there any difference in the performance (increase/decrease) of any job while execution?

4条回答
  •  清歌不尽
    2020-12-18 18:04

    i am using hadoop 2.2, and don't know how to set max input split size I would like to decrease this value, in order to create more mappers I tried updating yarn-site.xml, and but it does not work

    indeed, hadoop 2.2 /yarn does not take of none the following settings

    
    mapreduce.input.fileinputformat.split.minsize
    1
    
    
    mapreduce.input.fileinputformat.split.maxsiz e
    16777216
    
    
    
    mapred.min.split.size
    1
    
    
    mapred.max.split.size
    16777216
    
    

    best

提交回复
热议问题