data block size in HDFS, why 64MB?

后端 未结 8 1453
既然无缘
既然无缘 2020-11-28 21:43

The default data block size of HDFS/hadoop is 64MB. The block size in disk is generally 4KB. What does 64MB block size mean? ->Does it mean that the smallest unit of read fr

8条回答
  •  醉梦人生
    2020-11-28 22:32

    1. If block size was set to less than 64, there would be a huge number of blocks throughout the cluster, which causes NameNode to manage an enormous amount of metadata.
    2. Since we need a Mapper for each block, there would be a lot of Mappers, each processing a piece bit of data, which isn't efficient.

提交回复
热议问题