Iterate twice on values (MapReduce)

前端 未结 11 1116
轮回少年
轮回少年 2020-11-29 07:22

I receive an iterator as argument and I would like to iterate on values twice.

public void reduce(Pair key, Iterator          


        
11条回答
  •  星月不相逢
    2020-11-29 08:06

    Notice: if you use the cache list to cache the item, you should clone the item first then add to the cache. Otherwise you will find all the item is the same in the cache.

    The situation is caused by the memory optimization of the MapReduce, In the reduce method, the Iterable reuse the item instance, for more detail can find here

提交回复
热议问题