HBASE完全分布式安装

最新推荐文章于 2022-08-30 09:53:17 发布

Johnson8702

最新推荐文章于 2022-08-30 09:53:17 发布

阅读量682

点赞数 1

分类专栏： Linux系统大数据集群文章标签： HBase 完全分布式集群

本文链接：https://blog.csdn.net/Johnson8702/article/details/84783610

版权

大数据同时被 3 个专栏收录

18 篇文章 0 订阅

订阅专栏

Linux系统

15 篇文章 0 订阅

订阅专栏

集群

13 篇文章 0 订阅

订阅专栏

0、集群节点分配

Hadoop01：

Zookeeper+Hadoop+HBase

Hadoop02：

Zookeeper+HBase

Hadoop03：

Zookeeper+HBase

关于Hadoop2.0高可用集群搭建，请移步：https://blog.csdn.net/Johnson8702/article/details/84765141

1、安装和配置01节点的HBase

将压缩包上传到/home/software目录下，进行解压

[root@hadoop01 software]# pwd
/home/software
[root@hadoop01 software]# ll
总用量 86672
drwxr-xr-x 12 10021 10021     4096 12月 3 16:51 hadoop-2.7.1
-rw-r--r-- 1 root root 88739166 12月 4 10:02 hbase-0.98.17-hadoop2-bin.tar.gz
drwxr-xr-x 8 uucp    143     4096 3月 15 2017 jdk1.8
drwxr-xr-x 12 1000 1000     4096 12月 3 14:16 zookeeper
[root@hadoop01 software]# tar -xf hbase-0.98.17-hadoop2-bin.tar.gz
[root@hadoop01 software]# rm hbase-0.98.17-hadoop2-bin.tar.gz
rm：是否删除普通文件 "hbase-0.98.17-hadoop2-bin.tar.gz"？yes
[root@hadoop01 software]# ll
总用量 16
drwxr-xr-x 12 10021 10021 4096 12月 3 16:51 hadoop-2.7.1
drwxr-xr-x 7 root root 4096 12月 4 10:02 hbase-0.98.17-hadoop2
drwxr-xr-x 8 uucp    143 4096 3月 15 2017 jdk1.8
drwxr-xr-x 12 1000 1000 4096 12月 3 14:16 zookeeper
[root@hadoop01 software]# mv hbase-0.98.17-hadoop2/ hbase
[root@hadoop01 software]# ll
总用量 16
drwxr-xr-x 12 10021 10021 4096 12月 3 16:51 hadoop-2.7.1
drwxr-xr-x 7 root root 4096 12月 4 10:02 hbase
drwxr-xr-x 8 uucp    143 4096 3月 15 2017 jdk1.8
drwxr-xr-x 12 1000 1000 4096 12月 3 14:16 zookeeper
[root@hadoop01 software]#

进入hbase安装目录的conf目录，修改hbase-env.sh文件

[root@hadoop01 conf]# pwd
/home/software/hbase/conf
[root@hadoop01 conf]# ll
总用量 36
-rw-r--r-- 1 1000 1000 1811 1月 16 2016 hadoop-metrics2-hbase.properties
-rw-r--r-- 1 1000 1000 4023 1月 16 2016 hbase-env.cmd
-rw-r--r-- 1 1000 1000 7115 1月 16 2016 hbase-env.sh
-rw-r--r-- 1 1000 1000 2257 1月 16 2016 hbase-policy.xml
-rw-r--r-- 1 1000 1000 934 1月 16 2016 hbase-site.xml
-rw-r--r-- 1 1000 1000 4336 1月 16 2016 log4j.properties
-rw-r--r-- 1 1000 1000 10 1月 5 2016 regionservers
[root@hadoop01 conf]# vim hbase-env.sh

配置示例

第27行：export JAVA_HOME=/home/software/jdk1.8

配置jdk

第114行：export HBASE_PID_DIR=/var/hadoop/pids

配置pid的目录，默认存放在/tmp目录中，这是一个临时目录，可能会被清掉，导致hbase报错

第122行：export HBASE_MANAGES_ZK=false

修改Zookeeper和HBase的协调模式，HBase默认使用自带的zookeeper，如果需要使用外部的zookeeper，需要关闭自带的zookeeper

修改hbase-site.xml文件。配置开启完全分布模式

[root@hadoop01 conf]# pwd
/home/software/hbase/conf
[root@hadoop01 conf]# vim hbase-site.xml

在文件的末尾，配置如下：

<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://hadoop01:9000/hbase</value>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>hadoop01:2181,hadoop02:2181,hadoop03:2181</value>
</property>
</configuration>

修改regionservers文件，配置region服务器

每个主机名独占一行，hbase启动或关闭时会按照该配置顺序启动或关闭主机中的hbase

[root@hadoop01 conf]# pwd
/home/software/hbase/conf
[root@hadoop01 conf]# vim regionservers

文件配置如下：

hadoop01
hadoop02
hadoop03

2、配置02、03节点的hbase

将01节点配置好的hbase安装包远程发送到02、03节点的/home/software目录下

[root@hadoop01 software]# pwd
/home/software
[root@hadoop01 software]# scp -r hbase/ root@hadoop02:/home/software/

[root@hadoop01 software]# pwd
/home/software
[root@hadoop01 software]# scp -r hbase/ root@hadoop03:/home/software/

3、启动

启动01、02、03节点的Zookeeper服务

进入zookeeper的bin目录，执行：./zkServer.sh start

[root@hadoop01 bin]# pwd
/home/software/zookeeper/bin
[root@hadoop01 bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/software/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

[root@hadoop01 bin]# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/software/zookeeper/bin/../conf/zoo.cfg
Mode: follower
[root@hadoop01 bin]# jps
1472 Jps
1417 QuorumPeerMain
[root@hadoop01 bin]#

[root@hadoop02 bin]# pwd
/home/software/zookeeper/bin
[root@hadoop02 bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/software/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@hadoop02 bin]# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/software/zookeeper/bin/../conf/zoo.cfg
Mode: follower
[root@hadoop02 bin]# jps
1405 QuorumPeerMain
1695 Jps
[root@hadoop02 bin]#

[root@hadoop03 bin]# pwd
/home/software/zookeeper/bin
[root@hadoop03 bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/software/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@hadoop03 bin]# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/software/zookeeper/bin/../conf/zoo.cfg
Mode: leader
[root@hadoop03 bin]# jps
1705 Jps
1644 QuorumPeerMain
[root@hadoop03 bin]#

jps查看，01、02、03节点都会多出QuorumPeerMain进程。

启动Hadoop

在01节点任意目录执行：start-all.sh

[root@hadoop01 bin]# start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
18/12/04 10:47:38 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [hadoop01 hadoop02]
hadoop01: starting namenode, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-namenode-hadoop01.out
hadoop02: starting namenode, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-namenode-hadoop02.out
hadoop01: starting datanode, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-datanode-hadoop01.out
hadoop02: starting datanode, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-datanode-hadoop02.out
hadoop03: starting datanode, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-datanode-hadoop03.out
Starting journal nodes [hadoop01 hadoop02 hadoop03]
hadoop01: starting journalnode, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-journalnode-hadoop01.out
hadoop03: starting journalnode, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-journalnode-hadoop03.out
hadoop02: starting journalnode, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-journalnode-hadoop02.out
18/12/04 10:48:01 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting ZK Failover Controllers on NN hosts [hadoop01 hadoop02]
hadoop02: starting zkfc, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-zkfc-hadoop02.out
hadoop01: starting zkfc, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-zkfc-hadoop01.out
starting yarn daemons
starting resourcemanager, logging to /home/software/hadoop-2.7.1/logs/yarn-root-resourcemanager-hadoop01.out
hadoop01: starting nodemanager, logging to /home/software/hadoop-2.7.1/logs/yarn-root-nodemanager-hadoop01.out
hadoop03: starting nodemanager, logging to /home/software/hadoop-2.7.1/logs/yarn-root-nodemanager-hadoop03.out
hadoop02: starting nodemanager, logging to /home/software/hadoop-2.7.1/logs/yarn-root-nodemanager-hadoop02.out
[root@hadoop01 bin]#

从以上可以看出，分别在三个节点启动对应的进程，jps查看，三个节点会分别多出对应的进程

[root@hadoop01 bin]# jps
2288 DFSZKFailoverController
2497 NodeManager
2388 ResourceManager
2120 JournalNode
1417 QuorumPeerMain
1834 Nam
1931 DataNode
2829 Jps
[root@hadoop01 bin]#

创建/hbase目录（第一次启动hbase之前执行一次）

Hadoop启动之后，在hdfs文件系统中创建/hbase目录，要和hbase安装目录的conf目录下hbase-site.xml文件配置的目录路径一致

[root@hadoop01 bin]# hadoop fs -mkdir /hbase
18/12/04 10:57:03 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[root@hadoop01 bin]#

启动hbase

在01节点，进入HBase安装目录的bin目录下，执行：./start-hbase.sh

[root@hadoop01 bin]# pwd
/home/software/hbase/bin
[root@hadoop01 bin]# ./start-hbase.sh
starting master, logging to /home/software/hbase/bin/../logs/hbase-root-master-hadoop01.out
hadoop03: starting regionserver, logging to /home/software/hbase/bin/../logs/hbase-root-regionserver-hadoop03.out
hadoop02: starting regionserver, logging to /home/software/hbase/bin/../logs/hbase-root-regionserver-hadoop02.out
hadoop01: starting regionserver, logging to /home/software/hbase/bin/../logs/hbase-root-regionserver-hadoop01.out
[root@hadoop01 bin]#

jps查看，01节点多出HMaster和HRegionServer进程，02和03节点多出HRegionServer进程

web管理页面查看

通过浏览器访问：192.168.205.141:60010，通过web页面管理hbase

4、关闭

关闭hbase

在01节点，进入HBase安装目录的bin目录下，执行：./stop-hbase.sh

[root@hadoop01 bin]# pwd
/home/software/hbase/bin
[root@hadoop01 bin]# ./stop-hbase.sh
stopping hbase....................
[root@hadoop01 bin]#

jps查看，01节点减少HMaster和HRegionServer进程，02和03节点减少HRegionServer进程

关闭hadoop

在01节点任意目录执行：stop-all.sh

[root@hadoop01 bin]# stop-all.sh
This script is Deprecated. Instead use stop-dfs.sh and stop-yarn.sh
18/12/04 11:18:40 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Stopping namenodes on [hadoop01 hadoop02]
hadoop01: stopping namenode
hadoop02: stopping namenode
hadoop01: stopping datanode
hadoop02: stopping datanode
hadoop03: stopping datanode
Stopping journal nodes [hadoop01 hadoop02 hadoop03]
hadoop01: stopping journalnode
hadoop03: stopping journalnode
hadoop02: stopping journalnode
18/12/04 11:19:01 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Stopping ZK Failover Controllers on NN hosts [hadoop01 hadoop02]
hadoop01: stopping zkfc
hadoop02: stopping zkfc
stopping yarn daemons
stopping resourcemanager
hadoop03: stopping nodemanager
hadoop01: stopping nodemanager
hadoop02: stopping nodemanager
no proxyserver to stop
[root@hadoop01 bin]#

从以上可以看出，分别在三个节点关闭对应的进程，jps查看，三个节点会分别减少对应的进程

关闭01、02、03节点的Zookeeper服务

进入zookeeper的bin目录，执行：./zkServer.sh stop

[root@hadoop01 bin]# pwd
/home/software/zookeeper/bin
[root@hadoop01 bin]# ./zkServer.sh stop
ZooKeeper JMX enabled by default
Using config: /home/software/zookeeper/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
[root@hadoop01 bin]# jps
4454 Jps
[root@hadoop01 bin]#

[root@hadoop02 bin]# pwd
/home/software/zookeeper/bin
[root@hadoop02 bin]# ./zkServer.sh stop
ZooKeeper JMX enabled by default
Using config: /home/software/zookeeper/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
[root@hadoop02 bin]# jps
3020 Jps
[root@hadoop02 bin]#

[root@hadoop03 bin]# pwd
/home/software/zookeeper/bin
[root@hadoop03 bin]# ./zkServer.sh stop
ZooKeeper JMX enabled by default
Using config: /home/software/zookeeper/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
[root@hadoop03 bin]# jps
2541 Jps
[root@hadoop03 bin]#

jps查看，01、02、03节点都会减少QuorumPeerMain进程。

至此，HBase完全分布式安装完成！

关于HBase的SQL驱动---Phoenix，请移步：https://blog.csdn.net/Johnson8702/article/details/84873208