What is the use of grouping comparator in hadoop map reduce

后端 未结 4 744
感情败类
感情败类 2020-12-01 00:09

I would like to know why grouping comparator is used in secondary sort of mapreduce.

According to the definitive guide example of secondary sorting

We want t

4条回答
  •  猫巷女王i
    2020-12-01 01:11

    You need to introduce an intermediate key that is a composite of the year and temperature; partition on the natural key (the year) and introduce a comparator that will sort on the entire composite key. You're right that by partitioning on the year you'll get all the data for a year in the same reducer, so the comparator will effectively sort the data for each year by the temperature.

提交回复
热议问题