Hadoop 2.0 Name Node, Secondary Node and Checkpoint node for High Availability

Question

After reading Apache Hadoop documentation , there is a small confusion in understanding responsibilities of secondary node & check point node

I am clear on

Answer 1

NameNode- It is also known as Master node. Namenode stores meta-data i.e. number of blocks, their location, replicas and other details. This meta-data is available in memory in the master for faster retrieval of data. NameNode maintains and manages the slave nodes, and assigns tasks to them. It should be deployed on reliable hardware as it is the centerpiece of HDFS. Namenode holds its namespace using two files which are as follows:

FsImage: FsImage is an “Image file”. It contains the entire filesystem namespace and stored as a file in the namenode’s local file system.

EditLogs: It contains all the recent modifications made to the file system about the most recent FsImage.

Checkpoint node- Checkpoint node is a node which periodically creates checkpoints of the namespace. Checkpoint Node in Hadoop first downloads fsimage and edits from the active Namenode. Then it merges them (FsImage and edits) locally, and at last it uploads the new image back to the active NameNode. The checkpoint Node stores the latest checkpoint in a directory. It is structured in the same as the Namenode’s directory. It permits the checkpointed image to available for reading by the namenode.

Backup node- Backup node provides the same checkpointing functionality as the Checkpoint node. In Hadoop, Backup node keeps an in-memory, up-to-date copy of the file system namespace, which is always synchronized with the active NameNode state. The Backup node does not need to download fsimage and edits files from the active NameNode in order to create a checkpoint, as would be required with a Checkpoint node or Secondary Namenode, since it already has an up-to-date state of the namespace state in memory. The Backup node checkpoint process is more efficient as it only needs to save the namespace into the local fsimage file and reset edits. One Backup node is supported by the NameNode at a time. No checkpoint nodes may be registered if a Backup node is in use.