hbase

【HBase】（三）HBase 集群安装

由 无人久伴 提交于 2019-12-22 01:38:08
文章目录 一、前言 二、JDK / Hadoop 的选择 三、安装 1、zookeeper的安装 2、Hadoopd的安装 3、下载安装包 4、上传服务器并解压缩到指定目录 5、修改配置文件 （1）修改hbase-env.sh （2）修改hbase-site.xml （3）修改regionservers 6、配置环境变量 四、启动HBase集群 1、启动zookeeper集群 2、启动HBase 一、前言 1、HBase 依赖于 HDFS 做底层的数据存储 2、HBase 依赖于 MapReduce 做数据计算 3、HBase 依赖于 ZooKeeper 做服务协调 4、HBase源码是java编写的，安装需要依赖JDK 二、JDK / Hadoop 的选择 此处我们的hadoop版本用的的是2.6.0，HBase选择的版本是1.2.0 三、安装 1、zookeeper的安装 2、Hadoopd的安装 Hadoop 分布式安装 3、下载安装包 找到官网下载 hbase 安装包 hbase-1.2.0-bin.tar.gz，这里给大家提供一个下载地址： http://mirrors.hust.edu.cn/apache/hbase/ 4、上传服务器并解压缩到指定目录 5、修改配置文件 配置文件目录在安装包的conf文件夹中 （1）修改hbase-env.sh 1. export

Error: the constructor htable (configuration string) is deprecated

由 柔情痞子 提交于 2019-12-21 21:24:10
问题 I am using CDH 5.4.2 and trying to create Hbase Table have the following code snippet: Configuration conf = HBaseConfiguration.create(new Configuration()); HBaseAdmin hba = new <strike>HBaseAdmin</strike>(conf); if(!hba.tableExists(args[0])){ HTableDescriptor ht = new <strike>HTableDescriptor</strike> (args[0]); ht.addFamily(new HColumnDescriptor("sample")); There is a Deprecated error. How to avoid these warnings? Do I need to add any specific jars for CDH 5.4.2? 回答1: It's just a warning.

Create table in hbase

由 拜拜、爱过 提交于 2019-12-21 20:43:45
问题 I am new to hbase and hadoop. Anyhow I have succeeded in setting up a hadoop cluster which consists of 3 machines. Now I need some help on building up the database. I have a table "comments" contains fields: user id comments comments on comments (which can be more than one) and a status field to the same say, Could any one help me out to build the same using hbase/shell? 回答1: Here's some helpful HBase shell commands for you. Get help on format for creating a table in the shell. help 'create'

how to cancel command in GRUNT shell

由 一世执手 提交于 2019-12-21 20:31:07
问题 This is probably a more general question: Many tools in linux have their own shells. In my case, I use pig and hbase. Sometimes when you execute a command in the shell, it returns a lot of results and I want to cancel it. Let's say for example you do cat 'a.txt' and that file is huge. What's the best way to cancel it without exiting the shell. If I press Ctrl+c it'll exit the shell. 回答1: kill <job_id> will kill a mapreduce job `with the specified id. It's not exactly what you are looking for

【转载】HBase Region重点剖析

由 拥有回忆 提交于 2019-12-21 17:57:24
Region的概念 Region是HBase数据管理的基本单位。数据的move,数据的balance，数据的split，都是按照region来进行操作的。 region中存储这用户的真实数据，而为了管理这些数据，HBase使用了RegionSever来管理region。 一个表中可以包含一个或多个Region。 每个Region只能被一个RS（RegionServer）提供服务，RS可以同时服务多个Region，来自不同RS上的Region组合成表格的整体逻辑视图。 regionServer其实是hbase的服务，部署在一台物理服务器上，region有一点像 关系型数据的分区，数据存放在region中，当然region下面还有很多结构，确切来 说数据存放在memstore和hfile中。我们访问hbase的时候，先去hbase系统表查找 定位这条记录属于哪个region，然后定位到这个region属于哪个服务器，然后就到 哪个服务器里面查找对应region中的数据 每个region有三个主要要素: 它所属于哪张表 它所包含的的第一行(第一个region没有首行) 它所包含的最后一行(末一个region没有末行) 当表初写数据时，此时表只有一个region ,当随着数据的增多，region开始变大，等到它达到限定的阀值大小时，变化把region分裂为两个大小基本相同的region,

Connecting and Persisting to HBase

由 筅森魡賤 提交于 2019-12-21 17:39:34
问题 I just tried to connect to hbase which is part of the cloudera-vm using a java client. (192.168.56.102 is the inet ip of the vm) I use virtual box with host only network setting. So I can access the webUI of the hbase master @ http://192.168.56.102:60010/master.jsp Also my java client (worked well on the vm itself) established connection to 192.168.56.102:2181 But when it calls getMaster I get connection refused see log: 11/09/14 11:19:30 INFO zookeeper.ZooKeeper: Initiating client connection

hbase集群的启动,注意几个问题

由 女生的网名这么多〃 提交于 2019-12-21 16:41:01
1.hbase的改的会影响器他的组件的使用 故而, 在修改 hadoop的任何组件后, 一定要记得其它的组件也能受到影响, 一下是我在将hadoop的集群改了之后 , 再次运行hbase的时候, 就会发生异常, 原因是在连接namenode的时候,发生连接不到, 就是因为我改了之前的配置,将端口号改了, 没有去将配置文件在hbase中进行更新, 具体问如下: ##我今天起得hbase的集群时候,发生了 1.http://node3:60010/master-status HTTP ERROR 503 Problem accessing /master-status. Reason: Master not ready Powered by Jetty:// 2.http://node3:60030/rs-status The RegionServer is initializing! 在hbase-hadoop-master-node3.log中, 报异常是: 2016-10-12 09:45:04,616 DEBUG [main-EventThread] master.ActiveMasterManager: A master is now available 2016-10-12 09:59:10,484 WARN [master:node3:60000] retry

HBase与MapReduce的集成

由 折月煮酒 提交于 2019-12-21 10:04:59
HBase当中的数据最终都是存储在HDFS上面的，HBase天生的支持MR的操作，我们可以通过MR直接处理HBase当中的数据，并且MR可以将处理后的结果直接存储到HBase当中去 一、读取myuser这张表当中的数据写入到HBase的另外一张表当中去 读取HBase当中一张表的数据，然后将数据写入到HBase当中的另外一张表当中去。注意：我们可以使用TableMapper与TableReducer来实现从HBase当中读取与写入数据 将myuser这张表当中f1列族的name和age字段写入到myuser2这张表的f1列族当中去 1、创建myuser2这张表 hbase(main):010:0> create 'myuser2','f1' 2、创建maven工程，导入jar包 <repositories> <repository> <id>cloudera</id> <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url> </repository> </repositories> <dependencies> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-client</artifactId>

Why Hive is not supporting Stored Procedure?

由 时光怂恿深爱的人放手 提交于 2019-12-21 05:41:39
问题 Why hive is not supporting Stored procedure? If its not supporting then how we will handle Sp in Hive? have any alternate solution? (Because we have a already a data base is there in mssql) What about HBASE? Is it support SP? 回答1: First of all, Hadoop or Hive is NOT an alternative to your SQL DB. You must never consider either of these 2 to be used as a replacement of your RDBMS. Hive was developed just to provide warehousing capabilities on top of an existing Hadoop cluster keeping in mind

Create secondary index using coprocesor HBase

由 时间秒杀一切 提交于 2019-12-21 05:30:22
问题 I've been trying to write my own coprocessor that creates a secondary index using the prePut hook. To start, I've been simply trying to get a prePut coprocessor to work. So far I can have the coprocessor add to the put object passed to it. What i've found is that I cannot get the coprocessor to write to a row separate from what the passed in put object is writing to. Obviously to create a secondary index, I need to figure this one out. Below is the code for my coprocessor, but it doesn't work
订阅 hbase