用户提交任务到yarn报错

匿名 (未验证) 提交于 2019-12-02 23:34:01

用户提交任务到yarn时有可能遇到下面的错误:

1)

Requested user anything is not whitelisted and has id 980,which is below the minimum allowed 1000

这是因为yarn中配置min.user.id=1000,yarn认为id小于1000的是超级用户,yarn禁止超级用户提交任务;

Each account must have a user ID that is greater than or equal to 1000. In the /etc/hadoop/conf/taskcontroller.cfg file, the default setting for the banned.users property is mapred, hdfs, and bin to prevent jobs from being submitted via those user accounts. The default setting for the min.user.id property is 1000 to prevent jobs from being submitted with a user ID less than 1000, which are conventionally Unix super users.

改为更小即可

然后重启yarn

2)

Requested user hdfs is banned

这是因为yarn中配置banned.users=hdfs,yarn,mapred,bin,会阻止这几个用户提交任务

the default setting for the banned.users property is hdfs, yarn, mapred, and bin to prevent jobs from being submitted via those user accounts.

3)其他组件比如oozie无法获得提交到yarn上的application的状态

 2019-04-01 11:22:23,528 WARN org.apache.oozie.command.wf.ActionCheckXCommand: SERVER[cdp-test-server-04.bj] USER[impala] GROUP[-] TOKEN[] APP[test_sh_wf] JOB[0000000-190330171215585-oozie-oozi-W] ACTION[0000000-190330171215585-oozie-oozi-W@test_sh_action] Exception while executing check(). Error Code [JA017], Message[JA017: Could not lookup launched hadoop Job ID [job_1553937093273_0007] which was associated with  action [0000000-190330171215585-oozie-oozi-W@test_sh_action].  Failing this action!] org.apache.oozie.action.ActionExecutorException: JA017: Could not lookup launched hadoop Job ID [job_1553937093273_0007] which was associated with  action [0000000-190330171215585-oozie-oozi-W@test_sh_action].  Failing this action!         at org.apache.oozie.action.hadoop.JavaActionExecutor.check(JavaActionExecutor.java:1507)         at org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:182)         at org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:56)         at org.apache.oozie.command.XCommand.call(XCommand.java:286)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)

此时请检查jobhistory server,如果jobhistory server页面中没有application,也可以通过curl查看:

 # curl http:/localhost:19888/jobhistory -v ...             <script type="text/javascript">               var jobsTableData=[ ]             </script> ...

正常来说这个数组应该存放很多application,这个问题需要检查hdfs上/user/history及子目录是否存在以及权限

$ hdfs dfs -mkdir /user/history
$ hdfs dfs -chmod 777 /user/history

正常情况下jobhistory会从hdfs以下目录加载应用信息

/user/history/done/$year/$month/$day/
/user/history/done_intermediate/$user/

4)从jobhistory server查看application时报错

2019-04-01 16:04:16,115 WARN org.apache.oozie.command.wf.ActionCheckXCommand: SERVER[cdp-test-server-04.bj] USER[impala] GROUP[-] TOKEN[] APP[test_sh_wf] JOB[0000008-190330171215585-oozie-oozi-W] ACTION[0000008-190330171215585-oozie-oozi-W@test_sh_action] Exception while executing check(). Error Code [JA009], Message[JA009: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Could not load history file hdfs://hdfs:8020/user/history/done_intermediate/impala/job_1553937093273_0018-1554105820398-impala-oozie%3Alauncher%3AT%3Dshell%3AW%3Dtest_sh_wf%3AA%3Dtest_sh_acti-1554105833651-1-0-SUCCEEDED-root.users.impala-1554105825680.jhist
...
Caused by: org.apache.hadoop.security.AccessControlException: Permission denied: user=mapred, access=READ, inode="/user/history/done_intermediate/impala/job_1553937093273_0018-1554105820398-impala-oozie%3Alauncher%3AT%3Dshell%3AW%3Dtest_sh_wf%3AA%3Dtest_sh_acti-1554105833651-1-0-SUCCEEDED-root.users.impala-1554105825680.jhist":impala:supergroup:-rwxrwx---

hdfs中supergroup配置如下:

dfs.permissions.supergroup=supergroup
dfs.permissions.superusergroup=supergroup

解决方法:

# groupadd supergroup
# usermod -a -G supergroup mapred
# usermod -a -G supergroup hdfs
# usermod -a -G supergroup yarn

参考:https://www.cnblogs.com/barneywill/p/10628108.html

文章来源: https://blog.csdn.net/lvtula/article/details/90347052
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!