setup and cleanup methods of Mapper/Reducer in Hadoop MapReduce

前端 未结 5 1794
一生所求
一生所求 2020-12-24 03:10

Are setup and cleanup methods called in each mapper and reducer tasks respectively? Or are they called only once at the start of overall mapper and reducer jobs?

5条回答
  •  太阳男子
    2020-12-24 03:43

    It's called per Mapper task or Reducer task. Here is the hadoop code.

    public void run(Context context) throws IOException, InterruptedException {
        setup(context);
        try {
          while (context.nextKey()) {
            reduce(context.getCurrentKey(), context.getValues(), context);
          }
        } finally {
          cleanup(context);
        }
      }
    

提交回复
热议问题