Any command to get active namenode for nameservice in hadoop?

Question

The command:

hdfs haadmin -getServiceState machine-98

Works only if you know the machine name. Is there any command like:

h         


        

Answer 1

#!/usr/bin/python

import subprocess
import sys
import os, errno


def getActiveNameNode () :

    cmd_string="hdfs getconf -namenodes"
    process = subprocess.Popen(cmd_string, shell=True, stdout=subprocess.PIPE)
    out, err = process.communicate()
    NameNodes = out
    Value = NameNodes.split(" ")
    for val in Value :
        cmd_str="hadoop fs -test -e hdfs://"+val
        process = subprocess.Popen(cmd_str, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        out, err = process.communicate()
        if (err != "") :
            return val

def main():

    out = getActiveNameNode()
    print(out)

if __name__ == '__main__':
    main()

Answer 2

To print out the namenodes use this command:

hdfs getconf -namenodes

To print out the secondary namenodes:

hdfs getconf -secondaryNameNodes

To print out the backup namenodes:

hdfs getconf -backupNodes

Note: These commands were tested using Hadoop 2.4.0.

Update 10-31-2014:

Here is a python script that will read the NameNodes involved in Hadoop HA from the config file and determine which of them is active by using the hdfs haadmin command. This script is not fully tested as I do not have HA configured. Only tested the parsing using a sample file based on the Hadoop HA Documentation. Feel free to use and modify as needed.

#!/usr/bin/env python
# coding: UTF-8
import xml.etree.ElementTree as ET
import subprocess as SP
if __name__ == "__main__":
    hdfsSiteConfigFile = "/etc/hadoop/conf/hdfs-site.xml"

    tree = ET.parse(hdfsSiteConfigFile)
    root = tree.getroot()
    hasHadoopHAElement = False
    activeNameNode = None
    for property in root:
        if "dfs.ha.namenodes" in property.find("name").text:
            hasHadoopHAElement = True
            nameserviceId = property.find("name").text[len("dfs.ha.namenodes")+1:]
            nameNodes = property.find("value").text.split(",")
            for node in nameNodes:
                #get the namenode machine address then check if it is active node
                for n in root:
                    prefix = "dfs.namenode.rpc-address." + nameserviceId + "."
                    elementText = n.find("name").text
                    if prefix in elementText:
                        nodeAddress = n.find("value").text.split(":")[0]                

                        args = ["hdfs haadmin -getServiceState " + node]  
                        p = SP.Popen(args, shell=True, stdout=SP.PIPE, stderr=SP.PIPE)

                        for line in p.stdout.readlines():
                            if "active" in line.lower():
                                print "Active NameNode: " + node
                                break;
                        for err in p.stderr.readlines():
                            print "Error executing Hadoop HA command: ",err
            break            
    if not hasHadoopHAElement:
        print "Hadoop High-Availability configuration not found!"

Answer 3

You can do it in bash with hdfs cli calls, too. With the noted caveat that this takes a bit more time since it's a few calls to the API in succession, but this may be preferable to using a python script for some.

This was tested with Hadoop 2.6.0

get_active_nn(){
   ha_name=$1 #Needs the NameServiceID
   ha_ns_nodes=$(hdfs getconf -confKey dfs.ha.namenodes.${ha_name})
   active=""
   for node in $(echo ${ha_ns_nodes//,/ }); do
     state=$(hdfs haadmin -getServiceState $node)
     if [ "$state" == "active" ]; then
       active=$(hdfs getconf -confKey dfs.namenode.rpc-address.${ha_name}.${node})
       break
     fi
   done
   if [ -z "$active" ]; then
     >&2 echo "ERROR: no active namenode found for ${ha_name}"
     exit 1
   else
     echo $active
   fi
}

Answer 4

In HDFS 2.6.0 the one that worked for me

ubuntu@platform2:~$ hdfs getconf -confKey dfs.ha.namenodes.arkin-platform-cluster
nn1,nn2
ubuntu@platform2:~$ sudo -u hdfs hdfs haadmin -getServiceState nn1
standby
ubuntu@platform2:~$ sudo -u hdfs hdfs haadmin -getServiceState nn2
active