HBase2.2.0基于Hadoop3.1.3的安装配置
一、 安装部署HBase组件
1.1 准备HBase组件
Hbase下载链接:https://repo.huaweicloud.com/apache/hbase/2.2.0/
------------------------若想要其他版本把链接尾部的2.2.0/删除掉即可
以下均在在root用户下进行操作,如果想用其他用户,务必在解压文件前切换用户,以免因为文件所属用户,导致权限问题错误
在hadoop105上准备HBase组件。
步骤 1解压 HBase 软件包
在hadoop105节点解压 HBase 软件包至/opt路径下
[root@hadoop105 software]# tar -zxvf /software/hbase-2.2.0-bin.tar.gz -C /opt/module
[root@hadoop105 opt]# ls /opt | grep hbase
hbase-2.2.0
步骤 2修改hbase-env.sh文件
将hbase-env.sh文件的JAVA_HOME变量修改为当前操作系统正确变量:
vim /opt/module/hbase-2.2.0/conf/hbase-env.sh
在最后一行补充以下内容:
export JAVA_HOME=/opt/module/jdk1.8.0_212
export HBASE_MANAGES_ZK=true
步骤 3修改hbase-site.xml配置文件
vim /opt/module/hbase-2.2.0/conf/hbase-site.xml
将<configuration>
至</configuration>
中的内容替换为如下:
<property>
<name>hbase.rootdir</name>
<value>hdfs://hadoop105:8020/hbase</value>
</property>
<property>
<name>zookeeper.session.timeout</name>
<value>120000</value>
</property>
<property>
<name>hbase.zookeeper.property.tickTime</name>
<value>6000</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/opt/module/hbase-2.2.0/data/zookeeper</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>hadoop105,hadoop106,hadoop107</value>
</property>
<property>
<name>hbase.tmp.dir</name>
<value>/opt/module/hbase-2.2.0/tmp</value>
</property>
<property>
<name>hbase.wal.provider</name>
<value>org.apache.hadoop.hbase.wal.FSHLogProvider</value>
</property>
<property>
<name>hbase.wal.dir</name>
<value>hdfs://hadoop105:8020/hbase</value>
</property>
<property>
<name>hbase.client.write.buffer</name>
<value>5242880</value>
</property>
<property>
<name>hbase.regionserver.handler.count</name>
<value>200</value>
</property>
<property>
<name>hbase.hstore.compaction.min</name>
<value>6</value>
</property>
<property>
<name>hbase.hregion.memstore.block.multiplier</name>
<value>16</value>
</property>
<property>
<name>hfile.block.cache.size</name>
<value>0.2</value>
</property>
<!--hbase.unsafe.stream.capability.enforce这个属性的设置,是为了避免出现启动错误。
也就是说,如果没有设置hbase.unsafe.stream.capability.enforce为false,
那么,在启动HBase以后,会出现无法找到HMaster进程的错误,
启动后查看系统启动日志(/usr/local/hbase/logs/hbase-hadoop-master-ubuntu.log),会发现错误。
-->
<property>
<name>hbase.unsafe.stream.capability.enforce</name>
<value>false</value>
</property>
步骤 4配置Regionserver
vim /opt/hbase-2.2.0/conf/regionservers
删除默认的localhost,然后添加内容如下为:
hadoop105
hadoop106
hadoop107
步骤 5同步Hadoop的配置
拷贝Hadoop的core-site.xml配置文件至hbase/conf目录中:
[root@hadoop105 opt]# cp /opt/module/hadoop-3.1.3/etc/hadoop/core-site.xml /opt/module/hbase-2.2.0/conf/
步骤 6替换旧版依赖包
HBase的2.2.0版本默认的Jar包是Hadoop 3.1.3 版本,我们需要替换成3.1.3版本的:
[root@hadoop105 opt]# rm -rf /opt/module/hbase-2.2.0/lib/hadoop-*
find /opt/ -name hadoop-annotations-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-auth-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-client-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-common-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-hdfs-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-mapreduce-client-app-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-mapreduce-client-common-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-mapreduce-client-core-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-mapreduce-client-hs-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-mapreduce-client-hs-plugins-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-mapreduce-client-jobclient-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-mapreduce-client-jobclient-3.1.3-tests.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-mapreduce-client-shuffle-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-yarn-api-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-yarn-applications-distributedshell-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-yarn-applications-unmanaged-am-launcher-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-yarn-client-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-yarn-common-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-yarn-server-applicationhistoryservice-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-yarn-server-common-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-yarn-server-nodemanager-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-yarn-server-resourcemanager-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-yarn-server-tests-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name hadoop-yarn-server-web-proxy-3.1.3.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
find /opt/ -name zookeeper-3.4.5.jar |xargs -I {} cp {} /opt/module/hbase-2.2.0/lib
步骤 7分发组件
在hadoop105中配置好HBase后,分发组件文件到hadoop106-107节点中:
for i in {6..7};do scp -r /opt/module/hbase-2.2.0 root@hadoop10${i}:/opt/module;done
拷贝完毕后,在 hadoop105~hadoop107 节点,均可查看到如下目录:
ls /opt/module | grep hbase
hbase-2.2.0
步骤 8配置HBase的环境变量,执行下面命令:
vim /etc/profile.d/my_env.sh
在文件末尾添加如下的内容:
#HBASE_HOME
export HBASE_HOME=/opt/module/hbase-2.2.0
export PATH=$PATH:$HBASE_HOME/bin
hadoop105~hadoop107 节点均配置好后,执行如下命令确保环境变量生效:
source /etc/profile
1.2 启动并验证HBase
步骤 9启动HBase
在 hd01 节点,执行如下命令启动 hbase
start-hbase.sh
步骤 10查看HBase的Java进程
在hadoop105节点执行jps
命令,存在HMaster
、HRegionServer
、HquorumPeer
进程:
[root@hadoop105 ~]# jps
8529 Jps
7971 HRegionServer
5507 ResourceManager
7732 HQuorumPeer
5096 NameNode
7816 HMaster
5321 SecondaryNameNode
在hadoop106节点执行jps
命令,存在HRegionServer
、HquorumPeer
进程:
[root@hd02 ~]# jps
3830 HRegionServer
2664 NodeManager
3738 HQuorumPeer
2540 DataNode
4159 Jps
在hd03节点执行jps命令,存在HRegionServer、HquorumPeer进程:
[root@hadoop107 ~]# jps
3730 HQuorumPeer
2531 DataNode
3828 HRegionServer
4198 Jps
2655 NodeManager
步骤 11访问HBaseWebUI
界面
访问路径为:http://hadoop105:16010
步骤 12在hd01节点执行HBase shell进入客户端。
hbase shell
……
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
Version 2.2.0, re1673bb0bbfea21d6e5dba73e013b09b8b49b89b, Tue Jul 10 17:26:48 CST 2018
Took 0.0030 seconds
hbase(main):001:0
上述结果表明,已经成功进入HBase shell客户端。