HDFS free space available command

Question

Is there a hdfs command to see available free space in hdfs. We can see that through browser at master:hdfsport in browser , but for some reason I can\'t access this and I n

Answer 1

Try this:

hdfs dfsadmin -report

With older versions of Hadoop, try this:

hadoop dfsadmin -report

Answer 2

Try this command:

hdfs dfsadmin -report

Answer 3

Hadoop version 1:

hadoop fs -df -h

OR

hadoop dfsadmin -report

---

Hadoop version 2:

hdfs dfs -df -h

OR

hadoop dfsadmin -report

Answer 4

Methods

1. dfsadmin

In newer versions of HDFS the hadoop CLI for dfsadmin is deprecated:

$ sudo -u hdfs hadoop dfsadmin -report
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

So you should be using only hdfs at this point. Additionally when on systems where sudo is required you run it like so:

$ sudo -u hdfs hdfs dfsadmin -report

2. fs -df

You have an additional method available via the fs module to hadoop as well:

$ hadoop fs -df -h

Example output

dfsadmin

Also to provide a more thorough answer here's what the output would look like from a single node installation.

$ sudo -u hdfs hdfs dfsadmin -report
Configured Capacity: 7504658432 (6.99 GB)
Present Capacity: 527142912 (502.72 MB)
DFS Remaining: 36921344 (35.21 MB)
DFS Used: 490221568 (467.51 MB)
DFS Used%: 93.00%
Under replicated blocks: 128
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0

-------------------------------------------------
Live datanodes (1):

Name: 192.168.114.48:50010 (host-192-168-114-48.td.local)
Hostname: host-192-168-114-48.td.local
Decommission Status : Normal
Configured Capacity: 7504658432 (6.99 GB)
DFS Used: 490221568 (467.51 MB)
Non DFS Used: 6977515520 (6.50 GB)
DFS Remaining: 36921344 (35.21 MB)
DFS Used%: 6.53%
DFS Remaining%: 0.49%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 2
Last contact: Thu Feb 04 13:35:04 EST 2016

In the above example, the HDFS HDD space has been 100% utilized.

fs -df

That same system with the -df subcommand from the fs module:

$ hadoop fs -df -h
Filesystem                                 Size     Used  Available  Use%
hdfs://host-192-168-114-48.td.local:8020  7.0 G  467.5 M     18.3 M    7%