大数据系列四：HBase

一.前言

HBase是KEYVALUE存储，面向列的多版本映射分布式数据库。
HBase架构简图如下：

不多介绍，对于非结构化数据，现在有很多解决方案，HBase除非历史遗留，一般不会是最适合的那个，适当了解即可。

二.安装配置

1.zookeeper
1.1 vi /etc/profile #in all zk nodes
#########################
export ZOOKEEPER_HOME=/usr/local/zookeeper-3.6.0/
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$ZOOKEEPER_HOME/bin

1.2 tar -xzvf #in all zk nodes 版本越新越好
#########################
[root@ipsnode1 local]# pwd
/usr/local
[root@ipsnode1 local]# tar -xzvf zookeeper-3.6.0.tar.gz

1.3 vi zoo.cfg #in all zk nodes
#########################
cp zoo_sample.cfg zoo.cfg
vi zoo.cfg
dataDir=/usr/local/zookeeper-3.6.0/data
server.1=ipsnode1:2888:3888 #2888集群内通信，3888选举leader,2181对客户端
server.2=ipsnode2:2888:3888
server.3=ipsnode3:2888:3888

1.4 vi myid #in all zk nodes
#########################
pwd
dataDir=/usr/local/zookeeper-3.6.0/data
vi myid #每个节点值都必须不同，如1,2,3

1.5 start #in all zookeeper nodes
#########################
[root@ipsnode1 bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.6.0/bin/…/conf/zoo.cfg
Starting zookeeper … STARTED

1.6 check
#########################
[root@ipsnode1 bin]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.6.0/bin/…/conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: follower

[root@ipsnode1 bin]# zkCli.sh -server ipsnode1:2181,ipsnode2:2181,ipsnode3:2181
[zk: ipsnode1:2181,ipsnode2:2181,ipsnode3:2181(CONNECTED) 30] ls /
[zookeeper]
[zk: ipsnode1:2181,ipsnode2:2181,ipsnode3:2181(CONNECTED) 31] ls /zookeeper
[config, quota]
[zk: ipsnode1:2181,ipsnode2:2181,ipsnode3:2181(CONNECTED) 32] ls /zookeeper/config
[]
[zk: ipsnode1:2181,ipsnode2:2181,ipsnode3:2181(CONNECTED) 33] ls /zookeeper/quota
[]
[zk: ipsnode1:2181,ipsnode2:2181,ipsnode3:2181(CONNECTED) 7] get /zookeeper/config
server.1=ipsnode1:2888:3888:participant
server.2=ipsnode2:2888:3888:participant
server.3=ipsnode3:2888:3888:participant
version=0
[zk: ipsnode1:2181,ipsnode2:2181,ipsnode3:2181(CONNECTED) 37] get /zookeeper/quota

2.HBASE
2.1 下载正确版本，查看hadoop&hbase支持矩阵
#########################
https://hbase.apache.org/book.html#basic.prerequisites
下载&解压到指定目录即可。

2.2 vi /etc/profile
#########################
export HBASE_HOME=/usr/local/hbase-2.1.10
export PATH=$HBASE_HOME/bin:$PATH

2.3 vi hbase-env.sh
#########################
export JAVA_HOME=/usr/java/jdk1.8.0_251-amd64
#不使用自带的ZK，因为已经有了一个zookeeper集群，直接使用就行了
export HBASE_MANAGES_ZK=false
export HBASE_HEAPSIZE=4G #

2.4 vi hbase-site.xml
#########################

<configuration>
<!-- 在hdfs的存储rootdir -->
  <property>
    <name>hbase.rootdir</name>
    <value>hdfs://ipsnode1:9000</value> <!-- import,referenced from hadoop core-site.xml -->
  </property>
<!-- zk地址-->
<property>
  <name>hbase.zookeeper.quorum</name>
  <value>ipsnode1,ipsnode2,ipsnode3</value>
</property>
<!-- 分布式模式 -->
<property>
  <name>hbase.cluster.distributed</name>
  <value>true</value>
</property>
 <property>
        <name>hbase.master.info.port</name>
        <value>16010</value>
 </property>
<!-- 解决htrace jar包错误 -->
<property>
  <name>hbase.unsafe.stream.capability.enforce</name>
  <value>false</value>
</property>
</configuration>

2.5 vi regionservers
#########################
ipsnode1
ipsnode2
ipsnode3

2.6 vi backup-masters
#########################
ipsnode2

2.7 scp hbase to others node
#########################

2.8 start
#########################
start-hbase.sh

2.9 manage
#########################
http://192.168.100.101:16010/master-status

1…/hbase-daemon.sh stop regionserver RegionServer

2.if hbase error,please close hbase,delete hdfs’s hbase files,start hbase

三.测试

SQL:
create ‘user_info’, ‘base_info’, ‘ext_info’
–insert
put ‘user_info’, ‘1’, ‘base_info:name’, ‘a’
put ‘user_info’, ‘1’, ‘base_info:tel’, ‘123’
put ‘user_info’, ‘2’, ‘base_info:name’, ‘b’
put ‘user_info’, ‘2’, ‘ext_info:addr’, ‘bj’
对比结构化数据，大概如下：