Combiner Implementation and internal working

Question

I want to use a combiner in my MR code say WordCount.

How should I implement it?

What sort of data is being passed to the reducer from the combiner?

Answer 1

the combiner is doing the same work as reducer ,it can implement the reducer interface and over ride it's reduce method.if you use combiner,smaller amount of the network bandwidth is enough to transfer intermediate (o/p of mapper) to reducer.

you can use the same reduce method (belongs to your own reducer) code in combiner reduce method if your application used in reducer is obey both Commutative and Associative.

there is no rule to execute the Combiner even though you write the combiner for your MR(map reduce) application.to execute the combiner for sure the num of spills should be 3 at least.

for ex my mapper output is ,,,,,<34>. with out combiner ,my input to reducer is . with combiner ,i can pass input to reducer is like ,.