hadoop: difference between 0 reducer and identity reducer?

Question

I am just trying to confirm my understanding of difference between 0 reducer and identity reducer.

• 0 reducer means reduce step will be skipped and mapper outp

Answer 1

The main difference between "No Reducer" (mapred.reduce.tasks=0) and "Standard reducer" which is IdentityReducer (mapred.reduce.tasks=1 etc) is when you use "No reducer" there is no partitioning&shuffling processes after MAP stage. Therefore, in this case you will get 'pure' output from your mappers without any further processing. It helps for development and debugging puproses, but not only.