Hadoop fs lookup for block size?

前端 未结 5 852
悲哀的现实
悲哀的现实 2021-02-05 12:22

In Hadoop fs how to lookup the block size for a particular file?

I was primarily interested in a command line, something like:

hadoop fs ... hdfs://fs1.d         


        
5条回答
  •  眼角桃花
    2021-02-05 13:03

    The fsck commands in the other answers list the blocks and allow you to see the number of blocks. However, to see the actual block size in bytes with no extra cruft do:

    hadoop fs -stat %o /filename
    

    Default block size is:

    hdfs getconf -confKey dfs.blocksize
    

    Details about units

    The units for the block size are not documented in the hadoop fs -stat command, however, looking at the source line and the docs for the method it calls we can see it uses bytes and cannot report block sizes over about 9 exabytes.

    The units for the hdfs getconf command may not be bytes. It returns whatever string is being used for dfs.blocksize in the configuration file. (This is seen in the source for the final function and its indirect caller)

提交回复
热议问题