When do reduce tasks start in Hadoop?

前端 未结 8 841
深忆病人
深忆病人 2020-11-27 10:04

In Hadoop when do reduce tasks start? Do they start after a certain percentage (threshold) of mappers complete? If so, is this threshold fixed? What kind of threshold is typ

8条回答
  •  一向
    一向 (楼主)
    2020-11-27 10:08

    As much I understand Reduce phase start with the map phase and keep consuming the record from maps. However since there is sort and shuffle phase after the map phase all the outputs have to be sorted and sent to the reducer. So logically you can imagine that reduce phase starts only after map phase but actually for performance reason reducers are also initialized with the mappers.

提交回复
热议问题