Hadoop 2.5.0 on Mesos 0.21.0 with library 0.0.8 executor error

笑着哭i 提交于 2019-12-12 01:54:17

问题


The stderr logs the following while running a map-reduce job:

root@dbpc42:/tmp/mesos/slaves/20141201-225046-698725789-5050-19765-S24/frameworks/20141201-225046-698725789-5050-19765-0016/executors/executor_Task_Tracker_2/runs/latest# ls
hadoop-2.5.0-cdh5.2.0  hadoop-2.5.0-cdh5.2.0.tgz  stderr  stdout

Contents of stderr:

WARNING: Logging before InitGoogleLogging() is written to STDERR I1202 19:41:40.323521 7223 fetcher.cpp:76] Fetching URI 'hdfs://dbpc41:9000/hadoop-2.5.0-cdh5.2.0.tgz' I1202 19:41:40.323685 7223 fetcher.cpp:105] Downloading resource from 'hdfs://dbpc41:9000/hadoop-2.5.0-cdh5.2.0.tgz' to '/tmp/mesos/slaves/20141201-225046-698725789-5050-19765-S24/frameworks/20141201-225046-698725789-5050-19765-0016/executors/executor_Task_Tracker_0/runs/59c9a5ac-c376-4e9e-b9b7-63a24ae84dcd/hadoop-2.5.0-cdh5.2.0.tgz'

I1202 19:41:48.779220 7223 fetcher.cpp:64] Extracted resource '/tmp/mesos/slaves/20141201-225046-698725789-5050-19765-S24/frameworks/20141201-225046-698725789-5050-19765-0016/executors/executor_Task_Tracker_0/runs/59c9a5ac-c376-4e9e-b9b7-63a24ae84dcd/hadoop-2.5.0-cdh5.2.0.tgz' into '/tmp/mesos/slaves/20141201-225046-698725789-5050-19765-S24/frameworks/20141201-225046-698725789-5050-19765-0016/executors/executor_Task_Tracker_0/runs/59c9a5ac-c376-4e9e-b9b7-63a24ae84dcd'

sh: 1: cd: can't cd to hadoop-2.5.0-cdh5.2.0

sh: 1: ./bin/hadoop: not found

The stdout is empty.

The interesting part is, that when I run the slave on the same node where the master runs, the problem does not appear and tasks finish successfully. Only remote slaves introduce this problem. What could be the problem?


回答1:


The script placed by the JobTracker and ran by the executor was not able to access the directory. I've solved the problem by changing permissions recursively for directory /tmp to "free for all", 777.




回答2:


What you need to do is to give +x access wherever your mesos-work-dir is located. You don't need 777 recursively.



来源:https://stackoverflow.com/questions/27254666/hadoop-2-5-0-on-mesos-0-21-0-with-library-0-0-8-executor-error

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!