前言
关于hadoop完全分布式部署,我在https://blog.csdn.net/zisefeizhu/article/details/84317520已经有详细步骤
接https://blog.csdn.net/zisefeizhu/article/details/84317520继续部署hbase分布式
关于有关原理请关注这个大佬https://www.cnblogs.com/edisonchou/p/4440107.html
环境和此链接一致,在这链接出现的内容在此不在声明https://www.cnblogs.com/edisonchou/p/4440107.html
本次所需软件:zookeeper-3.4.9 hbase-1.2.4
网盘分享:
链接:https://pan.baidu.com/s/1KBfPO9D1UpTlwX5STVryVw
提取码:0xrp
开始正式部署
部署前的jps环境检查
[hadoop@hadoop01 ~]$ jps
1445 Jps
[hadoop@hadoop01 ~]$ /home/hadoop/hadoop/sbin/start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Starting namenodes on [hadoop01]
hadoop01: starting namenode, logging to /home/hadoop/hadoop-2.7.3/logs/hadoop-hadoop-namenode-hadoop01.out
hadoop02: starting datanode, logging to /home/hadoop/hadoop-2.7.3/logs/hadoop-hadoop-datanode-hadoop02.out
hadoop03: starting datanode, logging to /home/hadoop/hadoop-2.7.3/logs/hadoop-hadoop-datanode-hadoop03.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /home/hadoop/hadoop-2.7.3/logs/hadoop-hadoop-secondarynamenode-hadoop01.out
starting yarn daemons
starting resourcemanager, logging to /home/hadoop/hadoop-2.7.3/logs/yarn-hadoop-resourcemanager-hadoop01.out
hadoop02: starting nodemanager, logging to /home/hadoop/hadoop-2.7.3/logs/yarn-hadoop-nodemanager-hadoop02.out
hadoop03: starting nodemanager, logging to /home/hadoop/hadoop-2.7.3/logs/yarn-hadoop-nodemanager-hadoop03.out
[hadoop@hadoop01 ~]$ jps
1940 ResourceManager
1782 SecondaryNameNode
2199 Jps
1584 NameNode
[hadoop@hadoop02 ~]$ jps
1726 Jps
1568 NodeManager
1455 DataNode
[hadoop@hadoop03 ~]$ jps
1750 Jps
1593 NodeManager
1480 DataNode
Zookeeper安装配置
在hadoop01上上传zookeeper压缩文件
[hadoop@hadoop01 ~]$ rz
[hadoop@hadoop01 ~]$ ls |grep zookeeper
zookeeper-3.4.9.tar.gz
解压zookeeper文件
tar xf zookeeper-3.4.9.tar.gz
设置软链接
[hadoop@hadoop01 ~]$ ln -s zookeeper-3.4.9 zookeeper
配置环境变量
[hadoop@hadoop01 ~]$ vim ~/.bash_profile
PATH=$PATH:$HOME/.local/bin:$HOME/bin
#PATH=$PATH:$HOME/bin:/home/hadoop/java/bin
#java
export JAVA_HOME=/home/hadoop/java
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
#hadoop
export HADOOP_HOME=/home/hadoop/hadoop
export PATH=$PATH:$HADOOP_HOME/bin
export PATH
#zookeeper
export ZOOKEEPER=/home/hadoop/zookeeper
export PATH=$PATH:$ZOOKEEPER/bin
使修改生效
source ~/.bash_profile
[hadoop@hadoop01 ~]$ zk
zkCleanup.sh zkCli.cmd zkCli.sh zkEnv.cmd zkEnv.sh zkServer.cmd zkServer.sh
到zookeeper的conf目录下面,新增一个zoo.cfg文件
[hadoop@hadoop01 conf]$ cp zoo_sample.cfg zoo.cfg
修改zoo.cfg
dataDir=/home/hadoop/zookeeper/data
在zoo.cfg中添加
server.1=hadoop01:2888:3888
server.2=hadoop02:2888:3888
server.3=hadoop03:2888:3888
在hadoop01 的/home/hadoop/zookeeper 目录下 新建data目录,并在data目录下创建myid文件。
[hadoop@hadoop01 zookeeper]$ mkdir data
[hadoop@hadoop01 zookeeper]$ cd data/
[hadoop@hadoop01 data]$touch myid
将hadoop01上的有关配置拷贝给hadoop02,hadoop03【这里以hadoop02为例】
[hadoop@hadoop01 conf]$ scp -r /home/hadoop/zookeeper hadoop@hadoop02:/home/hadoop/
[hadoop@hadoop01 conf]$ scp ~/.bash hadoop@hadoop02:/home/hadoop/
[hadoop@hadoop01 conf]$ scp ~/.bash_profile hadoop@hadoop02:~/.
在hadoop02,hadoop03服务器上使环境变量生效
source ~/.bash_profile
在hadoop01 hadoop02 hadoop03的/home/hadoop/zookeeper/data/myid中分别添加1,2,3
注:此数字对应于zoo.cfg中的server号
在三台服务器上分别启动zookeeper,并查看启动状态,并使用jps查看【这里以hadoop01为例】
[hadoop@hadoop01 ~]$ zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/hadoop/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[hadoop@hadoop01 ~]$ zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/hadoop/zookeeper/bin/../conf/zoo.cfg
Mode: leader
[hadoop@hadoop01 ~]$ jps
2475 QuorumPeerMain
1940 ResourceManager
1782 SecondaryNameNode
2539 Jps
1584 NameNode
至此zookeeper配置完成!
HBase安装配置
【依旧在hadoop01上配置,然后拷贝】
[hadoop@hadoop01 ~]$ rz
[hadoop@hadoop01 ~]$ tar xf hbase-1.2.4-bin.tar.gz
[hadoop@hadoop01 ~]$ ln -s hbase-1.2.4 hbase
[hadoop@hadoop01 ~]$ ll | grep hbase
lrwxrwxrwx. 1 hadoop hadoop 11 11月 29 16:07 hbase -> hbase-1.2.4
drwxrwxr-x. 7 hadoop hadoop 160 11月 29 16:07 hbase-1.2.4
-rw-r--r--. 1 hadoop hadoop 104497899 11月 29 14:47 hbase-1.2.4-bin.tar.gz
修改环境变量
[hadoop@hadoop01 ~]$ vim ~/.bash_profile
#hbase
export HBASE_HOME=/home/hadoop/hbase
export PATH=$PATH:$HBASE_HOME/bin
使环境变量生效
[hadoop@hadoop01 ~]$ source ~/.bash_profile
拷贝给从服务器hadoop02,hadoop03【以hadoop02为例】
[hadoop@hadoop01 conf]$ scp ~/.bash_profile hadoop@hadoop02:~/.
在hadoop02,hadoop03服务器上使环境变量生效
source ~/.bash_profile
验证环境变量是否生效
[hadoop@hadoop01 ~]$ hbase
hbase hbase-common.sh hbase-daemon.sh hbase-jruby
hbase-cleanup.sh hbase-config.sh hbase-daemons.sh
进入hbase的conf目录,修改三个文件:hbase-env.sh hbase-site.xml 和regionservers
hbase-env.sh文件
#export JAVA_HOME=/usr/java/jdk1.6.0/
export JAVA_HOME=/home/hadoop/java
# Extra Java CLASSPATH elements. Optional.
# export HBASE_CLASSPATH=
# Seconds to sleep between slave commands. Unset by default. This
# can be useful in large clusters, where, e.g., slave rsyncs can
# otherwise arrive faster than the master can service them.
# export HBASE_SLAVE_SLEEP=0.1
# Tell HBase whether it should manage it's own instance of Zookeeper or not.
export HBASE_MANAGES_ZK=false
hbase-site.xml文件
[hadoop@hadoop01 conf]$ cat hbase-site.xml
<configuration>
<property>
<name>hbase.zookeeper.quorum</name>
<value>hadoop01,hadoop02,hadoop03</value>
<description>The directory shared by RegionServers.</description>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/home/hadoop/hbase/zookeeperdata</value>
<description>Property from ZooKeeper config zoo.cfg.
The directory where the snapshot is stored.
</description>
</property>
<property>
<name>hbase.tmp.dir</name>
<value>/home/hadoop/hbase/tmpdata</value>
</property>
<property>
<name>hbase.rootdir</name>
<value>hdfs://hadoop01:9000/hbase</value>
<description>The directory shared by RegionServers.</description>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
<description>The mode the cluster will be in. Possible values are
false: standalone and pseudo-distributed setups with managed Zookeeper
true: fully-distributed with unmanaged Zookeeper Quorum (see hbase-env.sh)
</description>
</property>
</configuration>
regionservers文件中添加服务器的ip或者hostname:【这里我添加hostname】
hadoop01
hadoop02
hadoop03
把hbase的整个文件夹拷贝到hadoop01 hadoop02服务器
[hadoop@hadoop01 hbase]$ jps
2475 QuorumPeerMain
1940 ResourceManager
1782 SecondaryNameNode
2859 Jps
1584 NameNode
[hadoop@hadoop01 hbase]$ jps
2475 QuorumPeerMain
1940 ResourceManager
1782 SecondaryNameNode
2859 Jps
1584 NameNode
[hadoop@hadoop01 hbase]$ start-hbase.sh
starting master, logging to /home/hadoop/hbase/logs/hbase-hadoop-master-hadoop01.out
hadoop03: starting regionserver, logging to /home/hadoop/hbase/bin/../logs/hbase-hadoop-regionserver-hadoop03.out
hadoop02: starting regionserver, logging to /home/hadoop/hbase/bin/../logs/hbase-hadoop-regionserver-hadoop02.out
hadoop01: starting regionserver, logging to /home/hadoop/hbase/bin/../logs/hbase-hadoop-regionserver-hadoop01.out
[hadoop@hadoop01 hbase]$ jps
3130 HRegionServer
2475 QuorumPeerMain
1940 ResourceManager
2990 HMaster
3178 Jps
1782 SecondaryNameNode
1584 NameNode
[hadoop@hadoop03 ~]$ jps
1922 QuorumPeerMain
1593 NodeManager
2361 Jps
2194 HRegionServer
1480 DataNode
至此hbase配置完成!!!
友情提示
启动顺序
hadoop-hdfs --->hadoop-yarn --->zookeeper -->hbase
-----zisefeizhu
-----2018/11/29/16:40