Hadoop Map Reduce reference static objects

前端 未结 3 1283
無奈伤痛
無奈伤痛 2021-01-07 03:38

I have a static object in my map reduce job class that I want to initialize once (in the main method), then call a function on it in every mapping. So I have this object, M

3条回答
  •  天命终不由人
    2021-01-07 04:03

    static object resides in memory. now your system is distributed one so object you had created is in memory of node on which your jobtracker is running not on other systems.

    now you cannot pass object from job to mapper because config is written as xml, but there is a workaround, Serialize your object into JSON and then put it as string in your configuration and in mappers deserialize this json object

    for job

    job.getConfiguration().set("some key", "json string")
    

    for mapper

    Configuration conf = context.getConfiguration();
    conf.get("some key");
    

提交回复
热议问题