Set hadoop system user for client embedded in Java webapp

后端 未结 3 1695
无人及你
无人及你 2020-11-28 10:23

I would like to submit MapReduce jobs from a java web application to a remote Hadoop cluster but am unable to specify which user the job should be submitted for. I would lik

3条回答
  •  遥遥无期
    2020-11-28 11:16

    I am able to resolve similar issue by using secure impersonation feature http://hadoop.apache.org/docs/stable1/Secure_Impersonation.html

    following is code snippet

        UserGroupInformation ugi = UserGroupInformation.createProxyUser("hduser", UserGroupInformation.getLoginUser()); 
    
        ugi.doAs(new PrivilegedExceptionAction() { 
        public Void run() throws Exception { 
          Configuration jobconf = new Configuration(); 
          jobconf.set("fs.default.name", "hdfs://server:hdfsport"); 
          jobconf.set("hadoop.job.ugi", "hduser"); 
          jobconf.set("mapred.job.tracker", "server:jobtracker port"); 
          String[] args = new String[] { "data/input", "data/output" }; 
          ToolRunner.run(jobconf, WordCount.class.newInstance(), args); 
          return null; 
        } });
    

    The remote (windows desktop host in my case) login user id should be added in core-site.xml as mentioned in above mentioned URL

提交回复
热议问题