historyserver not able to read log after enabling kerberos

旧城冷巷雨未停 提交于 2019-12-12 00:14:26

问题


I enable the Kerberos on the cluster and it is working fine. But due to some issue mapred user is not able to read and display log over JobHistory server. I check the logs of job history server and it giving access error as:

org.apache.hadoop.security.AccessControlException: Permission denied:user=mapred, access=READ_EXECUTE, inode="/user/history/done_intermediate/prakul":prakul:hadoop:drwxrwx---

as we can see the directory have access to hadoop group and mapred is in hadoop group, even then it is not able to read the logs. Similar error it is giving for /tmp/logs/ folder due to which no log was displayed on resource manager UI.

I verify over all machine that hadoop group contains mapred user on all machine:

[cloudera]# id mapred uid=491(mapred) gid=489(mapred) groups=489(mapred),496(hadoop)

I also kinit the mapred user and try to access manually to these directory, but mapred not able to access even when folder having 770 permission:

[root@mn0 cloudera]# hdfs dfs -ls /tmp/logs/prakul
ls: Permission denied: user=mapred, access=READ_EXECUTE, inode="/tmp/logs/prakul":prakul:hadoop:drwxrwx---

[root@mn0 cloudera]# hdfs dfs -ls /tmp/logs/
Found 8 items
drwxrwx--- - xyz hadoop 0 2016-06-14 19:19 /tmp/logs/xyz
drwxrwx--- - abc hadoop 0 2016-06-13 06:06 /tmp/logs/abc
drwxrwx--- - prakul hadoop 0 2016-06-10 04:47 /tmp/logs/prakul

[root@mn0 cloudera]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: mapred/mn0.eastus.cloudapp.azure.com@AD.COM
Valid starting Expires Service principal
06/27/16 01:07:32 06/27/16 11:07:32 krbtgt/AD.COM@AD.COM
renew until 07/04/16 01:07:32

If i give 777 permission to the directory then mapred is able to read and show log over UI as well as CLI.

Can any one know whether it is some Cloudera bug or there is some configuration issue due to which mapred not able to access the log even having full permission at group level?

I am using Cloudera 5.7 with Kerberos enabled.

thanks in advance


回答1:


I added a hadoop group in my AD server and made hdfs, yarn and mapred member of that group. After that I am able to view the logs.



来源:https://stackoverflow.com/questions/38114866/historyserver-not-able-to-read-log-after-enabling-kerberos

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!