Copy local data to hadoop hdfs error

孤街浪徒 提交于 2019-12-13 02:32:15

问题


I recently installed/configured hadoop and curently trying to run some tests. My problem is with copying local data to hdfs:

When I try to run

hdfs dfs -copyFromLocal /home/develop/test/ test

or any similar command, all i get is:

copyFromLocal: `test': No such file or directory

if I run ls, i get the same output:

develop@ubuntu:~$ hdfs dfs -ls
ls: `.': No such file or directory

I also tried to create the directory test with hdfs dfs -mkdir, but unsuccessful, what exactly am I missing ?


回答1:


Add slash in the HDFS path.

For example,

hdfs dfs -mkdir /test
hdfs dfs -copyFromLocal /home/develop/test/ /test



回答2:


If you want to copy a file from your local file system to HDFS, you need to first have that file in your local file system.

Create a file named test under /home/develop and then give the path of this file in the command.

You can even use this command too:

hdfs dfs -put /home/develop/test test



回答3:


After a fresh Hadoop installation ,you need to create user directory under /user on HDFS .

You are getting this error

develop@ubuntu:~$ hdfs dfs -ls
ls: `.': No such file or directory 

because you do not have any directory /user/develop on HDFS.

You can create user directory with following commands :

$> sudo su - hdfs -C "hadoop fs -mkdir /user/develop"
$> sudo su - hdfs -C "hadoop fs -chown -R develop:hadoop /user/develop"
$> sudo su - hdfs -C "hadoop fs -chmod -R 750 /user/develop"

You will still need to create test directory but if you use you copyFromLocal command then you need to create test directory in you HDFS user directory .so absolute path for your test dir will be /user/develop/test .



来源:https://stackoverflow.com/questions/32738837/copy-local-data-to-hadoop-hdfs-error

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!