namenode | 易学教程

hadoop2.6.0版本搭建伪分布式环境

阅读更多关于 hadoop2.6.0版本搭建伪分布式环境

由于个人需要在自己的笔记本上搭建hadoop伪分布环境，为了方便自己使用，如想看机器也看之前的一篇博客： hadoop2.6.0版本集群环境搭建一台虚拟机，配置信息如下：内存：1G，cpu：一个core,硬盘：15G 1、修改下主机名为master sudo vi /etc/sysconfig/network 修改结果后：重启电脑后再查看结果：修改主机名成功 2、修改hosts中的主机名：修改后： 3、配置SSH 进入.ssh目录并生成authorized_keys文件：授权.ssh/文件夹权限为700，authorized_keys文件权限为600(or 644)： ssh验证： 4、hadoop安装：去官网下载hadoop最新安装包： http://apache.fayea.com/hadoop/common/hadoop-2.6.0/ 下载并解压后：修改系统配置文件，修改~/.bash_profile文件，增加HADOPP_HOME的bin文件夹到PATH中，修改后使用source命令使配置文件生效：在hadoop目录下创建文件夹：接下来开始修改hadoop的配置文件，首先进入hadoop2.6配置文件夹：第一步修改配置文件hadoop-env.sh，加入"JAVA-HOME",如下所示：指定我们安装的“JAVA_HOME”：第二步修改配置文件

Namenode file quantity limit

阅读更多关于 Namenode file quantity limit

问题 Any one know how many bytes occupy per file in namenode of Hdfs? I want to estimate how many files can store in single namenode of 32G memory. 回答1: Each file or directory or block occupies about 150 bytes in the namenode memory. [1] So a cluster with a namenode with 32G RAM can support a maximum of (assuming namenode is the bottleneck) about 38 million files. (Each file will also take up a block, so each file takes 300 bytes in effect. I am also assuming 3x replication. So each file takes up

How does Hadoop Namenode failover process works?

阅读更多关于 How does Hadoop Namenode failover process works?

问题 Hadoop defintive guide says - Each Namenode runs a lightweight failover controller process whose job it is to monitor its Namenode for failures (using a simple heartbeat mechanism) and trigger a failover should a namenode fail. How come a namenode can run something to detect its own failure? Who sends heartbeat to whom? Where this process runs? How it detects namenode failure? To whom it notify for the transition? 回答1: From Apache docs The ZKFailoverController (ZKFC) is a new component which