putting a remote file into hadoop without copying it to local disk

前端 未结 4 417
猫巷女王i
猫巷女王i 2020-12-04 10:44

I am writing a shell script to put data into hadoop as soon as they are generated. I can ssh to my master node, copy the files to a folder over there and then put them into

4条回答
  •  感动是毒
    2020-12-04 11:07

    (untested)

    Since the node where you create your data has access to internet, then perhaps you could install hadoop client node software, then add it to the cluster - after normal hadoop fs -put, then disconnect and remove your temporary node - the hadoop system should then automatically make replication of your files blocks inside your hadoop cluster

提交回复
热议问题