Hadoop集群搭建Hbase集群

Habse介绍

HBase是一个分布式的、面向列的开源数据库，该技术来源于 Fay Chang 所撰写的Google论文“Bigtable：一个结构化数据的分布式存储系统”。就像Bigtable利用了Google文件系统（File System）所提供的分布式数据存储一样，HBase在Hadoop之上提供了类似于Bigtable的能力。HBase是Apache的Hadoop项目的子项目。HBase不同于一般的关系数据库，它是一个适合于非结构化数据存储的数据库。另一个不同的是HBase基于列的而不是基于行的模式。

HA Hadoop 高可用集群部署

HA Hadoop 高可用集群部署

部署环境介绍

1. 系统：CentOS Linux release 7.5.1804 (Core)
2. Hadoop：hadoop-2.7.3
3. Zookeeper：zookeeper-3.4.10
4. Jdk： jdk1.8.0_171
5. hbase-1.4.5

软件准备

wget http://archive.apache.org/dist/hbase/1.4.5/hbase-1.4.5-bin.tar.gz

集群部署规划

主机	地址	系统用户	服务进程	安装软件
hadoop-3	192.168.10.53	hadoop	Hmaster、HRegionServer	hbase
hadoop-4	192.168.10.54	hadoop	Hmaster、HRegionServer	hbase
hadoop-5	192.168.10.55	hadoop	Hmaster、HRegionServer	hbase

Hbase集群搭建

1. 将hbase包解压至指定目录并解压

[hadoop@hadoop-3 software]$ cd /data/software/
[hadoop@hadoop-3 software]$ tar -zxf hbase-1.4.5-bin.tar.gz -C /data/
[hadoop@hadoop-3 software]$ cd /data/
[hadoop@hadoop-3 data]$ mv hbase-1.4.5-bin hbase

2. 配置环境变量

[hadoop@hadoop-3 data]$ cat /etc/profile.d/hadoop.sh
export JAVA_HOME=/data/java/jdk1.8.0_171
export JRE_HOME=/data/java/jdk1.8.0_171/jre
export CLASSPATH=./:/data/java/jdk1.8.0_171/lib:/data/java/jdk1.8.0_171/jre/lib
export HADOOP_HOME=/data/hadoop
export ZOOKEEPER_HOME=/data/zookeeper
export HBASE_HOME=/data/hbase
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$ZOOKEEPER_HOME/bin:$HBASE_HOME/bin

3. 将搭建完成的Hadoop集群中core-site.xml文件和hdfs-site.xml复制到 /data/hbase/conf/ 目录下

[hadoop@hadoop-3 data]$ cp -a /data/hadoop/etc/hadoop/{core-site.xml,hdfs-site.xml} /data/hbase/conf/

4. 修改hbase-env.sh文件

[hadoop@hadoop-3 conf]$ grep -n 'JAVA_HOME' hbase-env.sh 
27:export JAVA_HOME=/data/java/jdk1.8.0_171
# 如果SSH端口有修改请添加端口号
102:# export HBASE_SSH_OPTS="-p 2222"
# 将其打开设置为false（默认注释）
[hadoop@hadoop-3 conf]$ grep -n 'HBASE_MANAGES_ZK' hbase-env.sh 
128:export HBASE_MANAGES_ZK=false

5. 修改hbase-site.xml文件

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
/**
 *
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership.  The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
-->
<configuration>
        <property>
                <name>hbase.rootdir</name>
                <value>hdfs://mycluster/hbase</value>
        </property>
        <property>
                <name>hbase.cluster.distributed</name>
                <value>true</value>
        </property>
        <property>
                <name>hbase.zookeeper.quorum</name>
                <value>hadoop-3:2181,hadoop-4:2181,hadoop-5:2181</value>
        </property>
        <property>
                <name>hbase.master.info.port</name>
                <value>60010</value>
        </property>
</configuration>

6. 修改/data/hbase/conf 路径下的regionservers文件

[hadoop@hadoop-3 conf]$ cat regionservers 
hadoop-3
hadoop-4
hadoop-5

7. 将配置好的Hbase传至其他节点

[hadoop@hadoop-3 conf]$ scp -r /data/hbase/ hadoop@hadoop-4:/data/
[hadoop@hadoop-3 conf]$ scp -r /data/hbase/ hadoop@hadoop-5:/data/

8. 启动Hbase集群

[hadoop@hadoop-3 conf]$ start-hbase.sh 
running master, logging to /data/hbase/logs/hbase-hadoop-master-hadoop-3.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
hadoop-3: running regionserver, logging to /data/hbase/logs/hbase-hadoop-regionserver-hadoop-3.out
hadoop-5: running regionserver, logging to /data/hbase/logs/hbase-hadoop-regionserver-hadoop-5.out
hadoop-4: running regionserver, logging to /data/hbase/logs/hbase-hadoop-regionserver-hadoop-4.out
hadoop-3: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
hadoop-3: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0

9. 为了保证集群的高可用性，需要启动多个HMaster

[hadoop@hadoop-4 ~]$ hbase-daemon.sh start master
running master, logging to /data/hbase/logs/hbase-hadoop-master-hadoop-4.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0

[hadoop@hadoop-5 ~]$ hbase-daemon.sh start master
running master, logging to /data/hbase/logs/hbase-hadoop-master-hadoop-5.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0

10. 验证集群是否启动成功

[hadoop@hadoop-3 conf]$ jps
1408 JournalNode
2226 HRegionServer
2088 HMaster
1226 QuorumPeerMain
1322 DataNode
1532 NodeManager
2588 Jps

[hadoop@hadoop-4 ~]$ jps
2368 HMaster
2528 Jps
1413 JournalNode
1222 QuorumPeerMain
1324 DataNode
1981 HRegionServer
1534 NodeManager

[hadoop@hadoop-5 ~]$ jps
1408 JournalNode
2402 Jps
2243 HMaster
1225 QuorumPeerMain
1322 DataNode
1531 NodeManager
1979 HRegionServer

11. 浏览器访问Hbase管理界面