HDFS Command Line Append

巧了我就是萌 提交于 2021-02-08 03:41:15

问题


Is there any way to append to a file on HDFS from command line like copying file:

hadoop fs -copyFromLocal <localsrc> URI

回答1:


This feature is implemented in Hadoop 2.3.0 as appendToFile with a syntax like:

hdfs dfs -appendToFile localfile /user/hadoop/hadoopfile

(it was first suggested in 2009 when the HDFS Append feature was being contemplated: https://issues.apache.org/jira/browse/HADOOP-6239 )




回答2:


cli doesn't support append, but httpfs and fuse both has support for appending files.

w301% ls -lA
total 0
-rw-r--r-- 1 hdfs supergroup 0 Nov 14 01:02 test.me
w301% cat test.me
w301% sync
w301% cat test.me
hello
w301% echo "more rows" >> test.me
w301% sync
w301% cat test.me 
hello
more rows

EDIT: but keep in mind, that only one client can append to file.



来源:https://stackoverflow.com/questions/13365604/hdfs-command-line-append

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!