0、集群节点分配
Hadoop01:
Zookeeper+Hadoop+HBase
Hadoop02:
Zookeeper+HBase
Hadoop03:
Zookeeper+HBase
关于Hadoop2.0高可用集群搭建,请移步:https://blog.csdn.net/Johnson8702/article/details/84765141
1、安装和配置01节点的HBase
- 将压缩包上传到/home/software目录下,进行解压
[root@hadoop01 software]# pwd /home/software [root@hadoop01 software]# ll 总用量 86672 drwxr-xr-x 12 10021 10021 4096 12月 3 16:51 hadoop-2.7.1 -rw-r--r-- 1 root root 88739166 12月 4 10:02 hbase-0.98.17-hadoop2-bin.tar.gz drwxr-xr-x 8 uucp 143 4096 3月 15 2017 jdk1.8 drwxr-xr-x 12 1000 1000 4096 12月 3 14:16 zookeeper [root@hadoop01 software]# tar -xf hbase-0.98.17-hadoop2-bin.tar.gz [root@hadoop01 software]# rm hbase-0.98.17-hadoop2-bin.tar.gz rm:是否删除普通文件 "hbase-0.98.17-hadoop2-bin.tar.gz"?yes [root@hadoop01 software]# ll 总用量 16 drwxr-xr-x 12 10021 10021 4096 12月 3 16:51 hadoop-2.7.1 drwxr-xr-x 7 root root 4096 12月 4 10:02 hbase-0.98.17-hadoop2 drwxr-xr-x 8 uucp 143 4096 3月 15 2017 jdk1.8 drwxr-xr-x 12 1000 1000 4096 12月 3 14:16 zookeeper [root@hadoop01 software]# mv hbase-0.98.17-hadoop2/ hbase [root@hadoop01 software]# ll 总用量 16 drwxr-xr-x 12 10021 10021 4096 12月 3 16:51 hadoop-2.7.1 drwxr-xr-x 7 root root 4096 12月 4 10:02 hbase drwxr-xr-x 8 uucp 143 4096 3月 15 2017 jdk1.8 drwxr-xr-x 12 1000 1000 4096 12月 3 14:16 zookeeper [root@hadoop01 software]# |
- 进入hbase安装目录的conf目录,修改hbase-env.sh文件
[root@hadoop01 conf]# pwd /home/software/hbase/conf [root@hadoop01 conf]# ll 总用量 36 -rw-r--r-- 1 1000 1000 1811 1月 16 2016 hadoop-metrics2-hbase.properties -rw-r--r-- 1 1000 1000 4023 1月 16 2016 hbase-env.cmd -rw-r--r-- 1 1000 1000 7115 1月 16 2016 hbase-env.sh -rw-r--r-- 1 1000 1000 2257 1月 16 2016 hbase-policy.xml -rw-r--r-- 1 1000 1000 934 1月 16 2016 hbase-site.xml -rw-r--r-- 1 1000 1000 4336 1月 16 2016 log4j.properties -rw-r--r-- 1 1000 1000 10 1月 5 2016 regionservers [root@hadoop01 conf]# vim hbase-env.sh |
配置示例
第27行:export JAVA_HOME=/home/software/jdk1.8
配置jdk
第114行:export HBASE_PID_DIR=/var/hadoop/pids
配置pid的目录,默认存放在/tmp目录中,这是一个临时目录,可能会被清掉,导致hbase报错
第122行:export HBASE_MANAGES_ZK=false
修改Zookeeper和HBase的协调模式,HBase默认使用自带的zookeeper,如果需要使用外部的zookeeper,需要关闭自带的zookeeper
- 修改hbase-site.xml文件。配置开启完全分布模式
[root@hadoop01 conf]# pwd /home/software/hbase/conf [root@hadoop01 conf]# vim hbase-site.xml |
在文件的末尾,配置如下:
<configuration> <property> <name>hbase.rootdir</name> <value>hdfs://hadoop01:9000/hbase</value> </property> <property> <name>dfs.replication</name> <value>1</value> </property> <property> <name>hbase.cluster.distributed</name> <value>true</value> </property> <property> <name>hbase.zookeeper.quorum</name> <value>hadoop01:2181,hadoop02:2181,hadoop03:2181</value> </property> </configuration> |
- 修改regionservers文件,配置region服务器
每个主机名独占一行,hbase启动或关闭时会按照该配置顺序启动或关闭主机中的hbase
[root@hadoop01 conf]# pwd /home/software/hbase/conf [root@hadoop01 conf]# vim regionservers |
文件配置如下:
hadoop01 hadoop02 hadoop03 |
2、配置02、03节点的hbase
- 将01节点配置好的hbase安装包远程发送到02、03节点的/home/software目录下
[root@hadoop01 software]# pwd /home/software [root@hadoop01 software]# scp -r hbase/ root@hadoop02:/home/software/ |
[root@hadoop01 software]# pwd /home/software [root@hadoop01 software]# scp -r hbase/ root@hadoop03:/home/software/ |
3、启动
- 启动01、02、03节点的Zookeeper服务
进入zookeeper的bin目录,执行:./zkServer.sh start
[root@hadoop01 bin]# pwd [root@hadoop01 bin]# ./zkServer.sh status |
[root@hadoop02 bin]# pwd /home/software/zookeeper/bin [root@hadoop02 bin]# ./zkServer.sh start ZooKeeper JMX enabled by default Using config: /home/software/zookeeper/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [root@hadoop02 bin]# ./zkServer.sh status ZooKeeper JMX enabled by default Using config: /home/software/zookeeper/bin/../conf/zoo.cfg Mode: follower [root@hadoop02 bin]# jps 1405 QuorumPeerMain 1695 Jps [root@hadoop02 bin]# |
[root@hadoop03 bin]# pwd /home/software/zookeeper/bin [root@hadoop03 bin]# ./zkServer.sh start ZooKeeper JMX enabled by default Using config: /home/software/zookeeper/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [root@hadoop03 bin]# ./zkServer.sh status ZooKeeper JMX enabled by default Using config: /home/software/zookeeper/bin/../conf/zoo.cfg Mode: leader [root@hadoop03 bin]# jps 1705 Jps 1644 QuorumPeerMain [root@hadoop03 bin]# |
jps查看,01、02、03节点都会多出QuorumPeerMain进程。
- 启动Hadoop
在01节点任意目录执行:start-all.sh
[root@hadoop01 bin]# start-all.sh This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh 18/12/04 10:47:38 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Starting namenodes on [hadoop01 hadoop02] hadoop01: starting namenode, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-namenode-hadoop01.out hadoop02: starting namenode, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-namenode-hadoop02.out hadoop01: starting datanode, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-datanode-hadoop01.out hadoop02: starting datanode, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-datanode-hadoop02.out hadoop03: starting datanode, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-datanode-hadoop03.out Starting journal nodes [hadoop01 hadoop02 hadoop03] hadoop01: starting journalnode, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-journalnode-hadoop01.out hadoop03: starting journalnode, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-journalnode-hadoop03.out hadoop02: starting journalnode, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-journalnode-hadoop02.out 18/12/04 10:48:01 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Starting ZK Failover Controllers on NN hosts [hadoop01 hadoop02] hadoop02: starting zkfc, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-zkfc-hadoop02.out hadoop01: starting zkfc, logging to /home/software/hadoop-2.7.1/logs/hadoop-root-zkfc-hadoop01.out starting yarn daemons starting resourcemanager, logging to /home/software/hadoop-2.7.1/logs/yarn-root-resourcemanager-hadoop01.out hadoop01: starting nodemanager, logging to /home/software/hadoop-2.7.1/logs/yarn-root-nodemanager-hadoop01.out hadoop03: starting nodemanager, logging to /home/software/hadoop-2.7.1/logs/yarn-root-nodemanager-hadoop03.out hadoop02: starting nodemanager, logging to /home/software/hadoop-2.7.1/logs/yarn-root-nodemanager-hadoop02.out [root@hadoop01 bin]# |
从以上可以看出,分别在三个节点启动对应的进程,jps查看,三个节点会分别多出对应的进程
[root@hadoop01 bin]# jps 2288 DFSZKFailoverController 2497 NodeManager 2388 ResourceManager 2120 JournalNode 1417 QuorumPeerMain 1834 Nam 1931 DataNode 2829 Jps [root@hadoop01 bin]# |
- 创建/hbase目录(第一次启动hbase之前执行一次)
Hadoop启动之后,在hdfs文件系统中创建/hbase目录,要和hbase安装目录的conf目录下hbase-site.xml文件配置的目录路径一致
[root@hadoop01 bin]# hadoop fs -mkdir /hbase 18/12/04 10:57:03 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable [root@hadoop01 bin]# |
- 启动hbase
在01节点,进入HBase安装目录的bin目录下,执行:./start-hbase.sh
[root@hadoop01 bin]# pwd /home/software/hbase/bin [root@hadoop01 bin]# ./start-hbase.sh starting master, logging to /home/software/hbase/bin/../logs/hbase-root-master-hadoop01.out hadoop03: starting regionserver, logging to /home/software/hbase/bin/../logs/hbase-root-regionserver-hadoop03.out hadoop02: starting regionserver, logging to /home/software/hbase/bin/../logs/hbase-root-regionserver-hadoop02.out hadoop01: starting regionserver, logging to /home/software/hbase/bin/../logs/hbase-root-regionserver-hadoop01.out [root@hadoop01 bin]# |
jps查看,01节点多出HMaster和HRegionServer进程,02和03节点多出HRegionServer进程
- web管理页面查看
通过浏览器访问:192.168.205.141:60010,通过web页面管理hbase
4、关闭
- 关闭hbase
在01节点,进入HBase安装目录的bin目录下,执行:./stop-hbase.sh
[root@hadoop01 bin]# pwd /home/software/hbase/bin [root@hadoop01 bin]# ./stop-hbase.sh stopping hbase.................... [root@hadoop01 bin]# |
jps查看,01节点减少HMaster和HRegionServer进程,02和03节点减少HRegionServer进程
- 关闭hadoop
在01节点任意目录执行:stop-all.sh
[root@hadoop01 bin]# stop-all.sh This script is Deprecated. Instead use stop-dfs.sh and stop-yarn.sh 18/12/04 11:18:40 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Stopping namenodes on [hadoop01 hadoop02] hadoop01: stopping namenode hadoop02: stopping namenode hadoop01: stopping datanode hadoop02: stopping datanode hadoop03: stopping datanode Stopping journal nodes [hadoop01 hadoop02 hadoop03] hadoop01: stopping journalnode hadoop03: stopping journalnode hadoop02: stopping journalnode 18/12/04 11:19:01 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Stopping ZK Failover Controllers on NN hosts [hadoop01 hadoop02] hadoop01: stopping zkfc hadoop02: stopping zkfc stopping yarn daemons stopping resourcemanager hadoop03: stopping nodemanager hadoop01: stopping nodemanager hadoop02: stopping nodemanager no proxyserver to stop [root@hadoop01 bin]# |
从以上可以看出,分别在三个节点关闭对应的进程,jps查看,三个节点会分别减少对应的进程
- 关闭01、02、03节点的Zookeeper服务
进入zookeeper的bin目录,执行:./zkServer.sh stop
[root@hadoop01 bin]# pwd /home/software/zookeeper/bin [root@hadoop01 bin]# ./zkServer.sh stop ZooKeeper JMX enabled by default Using config: /home/software/zookeeper/bin/../conf/zoo.cfg Stopping zookeeper ... STOPPED [root@hadoop01 bin]# jps 4454 Jps [root@hadoop01 bin]# |
[root@hadoop02 bin]# pwd /home/software/zookeeper/bin [root@hadoop02 bin]# ./zkServer.sh stop ZooKeeper JMX enabled by default Using config: /home/software/zookeeper/bin/../conf/zoo.cfg Stopping zookeeper ... STOPPED [root@hadoop02 bin]# jps 3020 Jps [root@hadoop02 bin]# |
[root@hadoop03 bin]# pwd /home/software/zookeeper/bin [root@hadoop03 bin]# ./zkServer.sh stop ZooKeeper JMX enabled by default Using config: /home/software/zookeeper/bin/../conf/zoo.cfg Stopping zookeeper ... STOPPED [root@hadoop03 bin]# jps 2541 Jps [root@hadoop03 bin]# |
jps查看,01、02、03节点都会减少QuorumPeerMain进程。
至此,HBase完全分布式安装完成!
关于HBase的SQL驱动---Phoenix,请移步:https://blog.csdn.net/Johnson8702/article/details/84873208