mrjob combiner not working python

荒凉一梦 提交于 2019-12-10 23:50:44

问题


Simple map combine reduce program: Map column-1 with value column-3 and append '+' in each mapper output of same key and append '-' after reduce output of same key.
input_1 and input_2 both files contain

a 1 2 3
a 4 5 6

Code is

from mrjob.job import MRJob
import re
import sys

class MRWordFreqCount(MRJob):

    def mapper(self, _, line):
        line=re.sub("\s\s+"," ",line)
        s1=line.split()
        yield(s1[0],s1[2])

    def combiner(self, accid, eventid):
        s="+"
        yield (accid, s.join(eventid))

    def reducer(self, accid, eventid):
         s="-"
         yield (accid, s.join(eventid))

if __name__ == '__main__':
    MRWordFreqCount.run()

After executing , python C:\Python27\map1.py C:\Python27\input_1.txt C:\Python27\input_2.txt result is "a" "2-2-5-5". Why combiner not works and append '+'.Expected output was:"a" "2+5-2+5". Thanks

来源:https://stackoverflow.com/questions/41118174/mrjob-combiner-not-working-python

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!