Hadoop: key and value are tab separated in the output file. how to do it semicolon-separated?

a 夏天 提交于 2019-11-28 07:41:12

Set the configuration property mapred.textoutputformat.separator to ";"

In lack of better documentation, here's what I've collected:

    setTextOutputFormatSeparator(final Job job, final String separator){
            final Configuration conf = job.getConfiguration(); //ensure accurate config ref

            conf.set("mapred.textoutputformat.separator", separator); //Prior to Hadoop 2 (YARN)
            conf.set("mapreduce.textoutputformat.separator", separator);  //Hadoop v2+ (YARN)
            conf.set("mapreduce.output.textoutputformat.separator", separator);
            conf.set("mapreduce.output.key.field.separator", separator);
            conf.set("mapred.textoutputformat.separatorText", separator); // ?
    }

you can use "KEY_VALUE_SEPERATOR " property of "KeyValueLineRecordReader" to specify a separator of your choice.

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!