Override hadoop's mapreduce.fileoutputcommitter.marksuccessfuljobs in oozie

后端 未结 3 999
盖世英雄少女心
盖世英雄少女心 2020-12-19 06:49

mapreduce.fileoutputcommitter.marksuccessfuljobs
false

I wan

3条回答
  •  轻奢々
    轻奢々 (楼主)
    2020-12-19 06:58

    Hive unfortunately overrides this capability by setting it's own NullOutputComitter:

    conf.setOutputCommitter(NullOutputCommitter.class);
    

    see

    src/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java
    src/shims/src/common-secure/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java
    

    Looks like you'll have to create the flag manually. We have filed HIVE-3700 for this.

提交回复
热议问题