Hadoop MapReduce intermediate output

前端 未结 2 843
感动是毒
感动是毒 2020-12-06 07:27

Is there a way to output to log the intermediate (Map Phase) output of a MapReduce Job without editing the Application? (The application is not mine, but the cluster is, and

2条回答
  •  挽巷
    挽巷 (楼主)
    2020-12-06 07:57

    keep.task.files.pattern parameter can be used to keep the intermediate files. The intermediate files have to be manually cleaned up once the Job has been completed. Since, this is a map/reduce task property, it has to be set in the configuration file and the jar file packaged again.

提交回复
热议问题