How To Automate Hadoop Trash Cleanup

青春壹個敷衍的年華 提交于 2019-12-23 12:38:09

问题


I can clear trash under my user folder by running hadoop fs -expunge This gets rid of files that are older than the fs.trash.interval value. Is there a for expunge to happen automatically to recover diskspace?

Also I see the following output when I run expunge

[cloudera@localhost conf]$ hadoop fs -expunge 14/07/17 15:43:54 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 1 minutes, Emptier interval = 0 minutes.

The emptier interval is 0 which suggests that automated clean up is turned off. Where is this value configured?


回答1:


The code suggests it is called fs.trash.interval.

EDIT: Sorry, misunderstood the question.

The emptier implementation itself is here, where we can see the relevant constant seems to be FS_TRASH_CHECKPOINT_INTERVAL_KEY.

Looking here reveals that key to be fs.trash.checkpoint.interval.

EDIT: Finally found the xml conf entry here.



来源:https://stackoverflow.com/questions/24814931/how-to-automate-hadoop-trash-cleanup

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!