Append data to existing file in HDFS Java

Question

I\'m having trouble to append data to an existing file in HDFS. I want that if the file exists then append a line, if not, create a new file with the name given.

Her

Answer 1

Actually, you can append to a HDFS file:

From the perspective of Client, append operation firstly calls append of DistributedFileSystem, this operation would return a stream object FSDataOutputStream out. If Client needs to append data to this file, it could calls out.write to write, and calls out.close to close.

I checked HDFS sources, there is DistributedFileSystem#append method:

 FSDataOutputStream append(Path f, final int bufferSize, final Progressable progress) throws IOException

For details, see presentation.

Also you can append through command line:

hdfs dfs -appendToFile  ... 

Add lines directly from stdin:

echo "Line-to-add" | hdfs dfs -appendToFile -