Hbase安装

Hbase安装与操作：前提，之前安装完ZOOKEEPER

一、HBASE安装:

1）上传，解压，修改环境变量
a）tar xzvf hbase-1.2.6-bin.tar.gz
b）cp -rp hbase-1.2.6 /usr/local/hbase
c）vi /etc/profile
export HBASE_HOME=/usr/local/hbase
export PATH=$PATH:$HBASE_HOME/bin

---

2）修改HBASE配置文件：/usr/local/hbase/conf
a）vi hbase-env.sh（:set nu）
a1）JAVA_HOME
27 # export JAVA_HOME=/usr/java/jdk1.6.0/
28
29 export JAVA_HOME=/usr/local/jdk1.8.0_45

a2）HBASE_CLASSPATH
31 # export HBASE_CLASSPATH=
32
33 export HBASE_CLASSPATH=/usr/local/hbase

a3）HBASE_PID_DIR
122 # export HBASE_PID_DIR=/var/hadoop/pids
123
124 export HBASE_PID_DIR=/var/hadoop/pids

a4）HBASE_MANAGES_ZK
131 # export HBASE_MANAGES_ZK=true
132
133 export HBASE_MANAGES_ZK=false

---

b）将zookeeper配置文件复制到HBASE的配置文件目录下（/usr/local/hbase/conf）
cp /usr/local/zookeeper/conf/zoo.cfg ./

---

c）hbase-site.xml：BIGPENG为主机名


hbase.master
60000

hbase.master.maxclockskew
180000

hbase.rootdir
hdfs://BIGPENG:9000/hbase

hbase.cluster.distributed
true

hbase.zookeeper.quorum
BIGPENG:2181

hbase.zookeeper.property.dataDir
/usr/local/hbase/zookeeper

hbase.tmp.dir
/var/hbase/tmp

---

d）regionservers （如果本地执行，此配置文件内为LOCALHOST）
Regionservers //是从机器的域名
slave1
slave2
slave3

---

3）启动Zookeeper:
[root@BIGPENG ~]# zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper … STARTED
[root@BIGPENG ~]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: standalone

---

4）启动Hbase服务：start-hbase.sh

5）验证服务：HBASE服务：（7608 HMaster、7719 HRegionServer）
[root@BIGPENG conf]# jps
4512 DataNode
4867 ResourceManager
7223 QuorumPeerMain
7719 HRegionServer
6936 RunJar
7608 HMaster
7817 Jps
4971 NodeManager
4685 SecondaryNameNode
4413 NameNode

6）进入网页控制台：192.168.175.10:16010

7）ZOOKEEPER：验证
[root@BIGPENG ~]# zkCli.sh
Connecting to localhost:2181
[zk: localhost:2181(CONNECTED) 0] ls /
[zookeeper, hbase]

---

8）进入HBASE：
[root@BIGPENG conf]# hbase shell
2018-05-30 10:56:03,068 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hbase/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
HBase Shell; enter ‘help’ for list of supported commands.
Type “exit” to leave the HBase Shell
Version 1.2.6, rUnknown, Mon May 29 02:25:32 CDT 2017

8.1）解决版本告警：
a）编辑：vi /usr/local/hbase/conf/log4j.properties
b）追加：log4j.logger.org.apache.hadoop.util.NativeCodeLoader=ERROR

8.2）与HADOOP（slf4j-log4j12-1.7.5.jar文件）冲突：
[root@BIGPENG conf]# hbase shell
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hbase/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
HBase Shell; enter ‘help’ for list of supported commands.
Type “exit” to leave the HBase Shell
Version 1.2.6, rUnknown, Mon May 29 02:25:32 CDT 2017

解决冲突：
[root@BIGPENG conf]# mv /usr/local/hbase/lib/slf4j-log4j12-1.7.5.jar /usr/local/hbase/lib/slf4j-log4j12-1.7.5.jar.bak

8.3）再次进入（以下信息为正常）：hbase shell
[root@BIGPENG conf]# hbase shell
HBase Shell; enter ‘help’ for list of supported commands.
Type “exit” to leave the HBase Shell
Version 1.2.6, rUnknown, Mon May 29 02:25:32 CDT 2017

hbase(main):001:0>

==================================================================================

二、HBASE操作：https://blog.csdn.net/kky2010_110/article/details/12616137

操作 ‘表名’,’主键’[，’列簇名’][，’列簇名：列名’]

1）创建表：表名（member），m_id和address和info都为列簇（非字段）
create ‘member’,’m_id’,’address’,’info’

---

2）查看所有表（等同show tables）：list
hbase(main):009:0* list
TABLE
member
1 row(s) in 0.1120 seconds

---

3）查看表结构（等同DESC TABLE_NAME）：describe ‘member’

hbase(main):010:0> describe ‘member’
Table member is ENABLED
member
COLUMN FAMILIES DESCRIPTION
{NAME => ‘address’, BLOOMFILTER => ‘ROW’, VERSIONS => ‘1’, IN_MEMORY => ‘false’, KEEP_DELETED_CELLS => ‘FALSE’, DATA_BLOC
K_ENCODING => ‘NONE’, TTL => ‘FOREVER’, COMPRESSION => ‘NONE’, MIN_VERSIONS => ‘0’, BLOCKCACHE => ‘true’, BLOCKSIZE => ‘6
5536’, REPLICATION_SCOPE => ‘0’}
{NAME => ‘info’, BLOOMFILTER => ‘ROW’, VERSIONS => ‘1’, IN_MEMORY => ‘false’, KEEP_DELETED_CELLS => ‘FALSE’, DATA_BLOCK_E
NCODING => ‘NONE’, TTL => ‘FOREVER’, COMPRESSION => ‘NONE’, MIN_VERSIONS => ‘0’, BLOCKCACHE => ‘true’, BLOCKSIZE => ‘6553
6’, REPLICATION_SCOPE => ‘0’}
{NAME => ‘m_id’, BLOOMFILTER => ‘ROW’, VERSIONS => ‘1’, IN_MEMORY => ‘false’, KEEP_DELETED_CELLS => ‘FALSE’, DATA_BLOCK_E
NCODING => ‘NONE’, TTL => ‘FOREVER’, COMPRESSION => ‘NONE’, MIN_VERSIONS => ‘0’, BLOCKCACHE => ‘true’, BLOCKSIZE => ‘6553
6’, REPLICATION_SCOPE => ‘0’}
3 row(s) in 0.3130 seconds

说明：
a）BLOOMFILTER => ‘ROW’（利用行进行zookeeper）
b）VERSIONS => ‘1’（允许一个版本，这里版本为在内存中保存修改的次数）
c）TTL => ‘FOREVER’（永久不失效，也可以设置时间）

---

4）查看表中数据（等同select * from member）：scan ‘member’
hbase(main):012:0* scan ‘member’
ROW COLUMN+CELL
0 row(s) in 0.1130 seconds

---

5）管理表结构（增删字段）
a）下线表：disable ‘member’
hbase(main):030:0> disable ‘member’
0 row(s) in 2.6660 seconds

b）查看表状态： is_enabled ‘member’
hbase(main):016:0> is_enabled ‘member’
false
0 row(s) in 0.0220 seconds

c）删除表中的列簇（HBASE大小写敏感）：alter ‘member’,{NAME=>’m_id’,METHOD=>’delete’}
hbase(main):026:0> alter ‘member’,{NAME=>’m_id’,METHOD=>’delete’}
Updating all regions with the new schema…
1/1 regions updated.
Done.
0 row(s) in 2.2800 seconds

d）上线表：enable ‘member’
hbase(main):027:0> enable ‘member’
0 row(s) in 1.3170 seconds

e）新增字段（可设置version）：alter ‘member’,{NAME=>’m_id’,VERSIONS=>’1’}
hbase(main):031:0> alter ‘member’,{NAME=>’m_id’,VERSIONS=>’1’}
Updating all regions with the new schema…
1/1 regions updated.
Done.
0 row(s) in 2.0290 seconds

---

6）DML管理：fujiapeng为ROW-KEY
a）插入数据：put’member’,’fujiapeng’,’info:age’,’28’
hbase(main):057:0* put’member’,’fujiapeng’,’info:age’,’28’
0 row(s) in 0.1070 seconds

---

b）修改数据（相当于覆盖）：put’member’,’fujiapeng’,’info:age’,’19’
hbase(main):062:0> put’member’,’fujiapeng’,’info:age’,’19’
0 row(s) in 0.0150 seconds

---

c）插入数据：put’member’,’fujiapeng’,’m_id:id’,’1’
hbase(main):059:0* put’member’,’fujiapeng’,’m_id:id’,’1’
0 row(s) in 0.0200 seconds

---

d）查询数据：scan ‘member’
hbase(main):063:0> scan ‘member’
ROW COLUMN+CELL
fujiapeng column=info:age, timestamp=1527652505721, value=19
fujiapeng column=m_id:id, timestamp=1527652458891, value=1
1 row(s) in 0.0410 seconds

---

e1）删除数据（指定ROW-KEY进行删除）：deleteall’member’,’fujiapeng1’
hbase(main):067:0> deleteall’member’,’fujiapeng1’
0 row(s) in 0.0570 seconds

e2）删除单条数据（指定ROW-KEY以及列簇对应的字段）
hbase(main):069:0> delete’member’,’fujiapeng’,’info:age’
0 row(s) in 0.0190 seconds

---

f）查询数据（可根据ROW-KEY，或者列簇，在或者列簇的字段）
hbase(main):085:0> get’member’,’fujiapeng’,’info’
COLUMN CELL
info:age timestamp=1527653156004, value=19
info:company timestamp=1527653243281, value=headline-today
2 row(s) in 0.0290 seconds

hbase(main):086:0> get’member’,’fujiapeng’,’info:age’
COLUMN CELL
info:age timestamp=1527653156004, value=19
1 row(s) in 0.0200 seconds

hbase(main):087:0> get’member’,’fujiapeng’
COLUMN CELL
info:age timestamp=1527653156004, value=19
info:company timestamp=1527653243281, value=headline-today
m_id:id timestamp=1527652458891, value=1
3 row(s) in 0.0180 seconds

---

g）通过timestamp来获取两个版本的数据（scan ‘member’）：get ‘member’,’fujiapeng’,{COLUMN=>’info:age’,TIMESTAMP=>1527653156004}
hbase(main):008:0> get ‘member’,’fujiapeng’,{COLUMN=>’info:age’,TIMESTAMP=>1527653156004}
COLUMN CELL
info:age timestamp=1527653156004, value=19
1 row(s) in 0.0700 seconds

---

7）HBase系统默认定义了两个缺省的namespace，hbase：系统内建表，包括namespace和meta表，default：用户建表时未指定namespace的表都创建在此

a）创建namespace：
hbase>create_namespace ‘ai_ns’

b）删除namespace：
hbase>drop_namespace ‘ai_ns’

c）查看namespace：
hbase>describe_namespace ‘ai_ns’

d）列出所有namespace
hbase>list_namespace

e）在namespace下创建表
hbase>create ‘ai_ns:testtable’,’fm1’

f）查看namespace下的表
hbase>list_namespace_tables ‘ai_ns’

---

8）授权：HBase namespace特性是对表资源进行隔离的一种技术，隔离技术决定了HBase能否实现资源统一化管理的关键，提高了整体的安全性。
8.1）具备Create权限的namespace Admin可以对表创建和删除、生成和恢复快照
8.2）具备Admin权限的namespace Admin可以对表splits或major compactions

a）授权tenant-A用户对ai_ns下的写权限
hbase>grant ‘tenant-A’ ‘W’ ‘@ai_ns’

b）回收tenant-A用户对ai_ns的所有权限
hbase>revoke ‘tenant-A”@ai_ns’

c）当前用户：hbase
hbase>namespace_create ‘hbase_perf’
hbase>grant ‘mike’, ‘W’, ‘@hbase_perf’

d）当前用户：mike
hbase>create ‘hbase_perf.table20’, ‘family1’
hbase>create ‘hbase_perf.table50’, ‘family1’

e）说明：mike创建了两张表table20和table50，同时成为这两张表的owner，意味着有’RWXCA’权限，此时，mike团队的另一名成员alice也需要获得hbase_perf下的权限，hbase管理员操作如下，当前用户：hbase
hbase>grant ‘alice’, ‘W’, ‘@hbase_perf’

f）此时alice可以在hbase_perf下创建表，但是无法读、写、修改和删除hbase_perf下已存在的表，当前用户：alice
hbase>scan ‘hbase_perf:table20’

g）报错AccessDeniedException，如果希望alice可以访问已经存在的表，则hbase管理员操作如下，当前用户：hbase
hbase>grant ‘alice’, ‘RW’, ‘hbase_perf.table20’
hbase>grant ‘alice’, ‘RW’, ‘hbase_perf.table50’

h）在HBase中启用授权机制（配置完成后需要重启HBase集群）：hbase-site.xml

hbase.security.authorization
true


hbase.coprocessor.master.classes
org.apache.hadoop.hbase.security.access.AccessController


hbase.coprocessor.region.classes
org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.AccessController

=====================================================================================

三、API

JAVA

Configuration conf = HBaseConfiguration.create();
HBaseAdmin admin = new HBaseAdmin(conf);

//create namespace named “ns”
admin.createNamespace(NamespaceDescriptor.create(“ns”).build());

admin.close();

注意以下几点：
1）必须将HBase集群的hbase-site.xml文件添加进工程的classpath中，或者通过Configuration对象设置相关属性，否则程序获取不到集群相关信息，也就无法找到集群，运行程序时会报错。
2）命名空间一般在建模阶段通过命令行创建，创建的机会不多。
3）创建HBaseAdmin对象时就已经建立了客户端程序与HBase集群的connection，所以在程序执行完成后，务必通过admin.close()，关闭connection；