About Hadoop/HDFS file splitting

后端 未结 3 2036
你的背包
你的背包 2020-12-07 23:41

Want to just confirm on following. Please verify if this is correct: 1. As per my understanding when we copy a file into HDFS, that is the point when file (assuming its size

3条回答
  •  不思量自难忘°
    2020-12-07 23:48

    Yes, file contents are split into chunks when the file is copied into the HDFS. The block size is configurable, and if it is say 128 MB, then whole 128 MB would be one block, not 2 blocks of 64 MB separately.Also it is not necessary that each chunk of a file is stored on a separate datanode.A datanode may have more than one chunk of a particular file.And a particular chunk may be present in more than one datanodes based upon the replication factor.

提交回复
热议问题