I get multiple small files into my input directory which I want to merge into a single file without using the local file system or writing mapreds. Is there a way I could do
All the solutions are equivalent to doing a
hadoop fs -cat [dir]/* > tmp_local_file hadoop fs -copyFromLocal tmp_local_file
it only means that the local m/c I/O is on the critical path of data transfer.