Hadoop DistributedCache is deprecated - what is the preferred API?

前端 未结 6 1269
情深已故
情深已故 2020-11-28 04:14

My map tasks need some configuration data, which I would like to distribute via the Distributed Cache.

The Hadoop MapReduce Tutorial shows the usage of the Distribut

6条回答
  •  北荒
    北荒 (楼主)
    2020-11-28 05:05

    The APIs for the Distributed Cache can be found in the Job class itself. Check the documentation here: http://hadoop.apache.org/docs/stable2/api/org/apache/hadoop/mapreduce/Job.html The code should be something like

    Job job = new Job();
    ...
    job.addCacheFile(new Path(filename).toUri());
    

    In your mapper code:

    Path[] localPaths = context.getLocalCacheFiles();
    ...
    

提交回复
热议问题