集群启动顺序
- ZK集群
- Hadoop集群
Hadoop HA集群搭建
集群架构
bigdata01 | bigdata02 | bigdata03 |
---|---|---|
namenode1 | namenode2 | |
datanode | datanode | datanode |
journal node | journal node | journal node |
zkfc | zkfc | zkfc |
resourcemanager1 | resourcemaneger2 | |
nodemanager | nodemanager | nodemanager |
zk01 | zk02 | zk03 |
HMaster | HMaster | |
HRegionServer | HRegionServer | HRegionServer |
集群搭建
# 每一台机器都需要进行修改,配置java环境,关闭防火墙
# 以第一台机器为例
1.修改静态ip地址
[root@bigdata01 ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens37
2.重启网卡
[root@bigdata01 ~]# systemctl restart network
3.host映射
[root@bigdata01 ~]# vim /etc/hosts
192.168.24.21 bigdata01
192.168.24.22 bigdata02
192.168.24.23 bigdata03
4.关闭防火墙
# 关闭防火墙
[root@bigdata01 ~]# systemctl stop firewalld
# 禁止防火墙开机自启
[root@bigdata01 ~]# systemctl disable firewalld.service
5.配置ssh免密登录
[root@bigdata01 ~]# ssh-keygen -t rsa
[root@bigdata01 ~]# ssh-copy-id bigdata02
[root@bigdata01 ~]# ssh-copy-id bigdata03
6.配置Hadoop环境变量
[root@bigdata01 ~]# vim .bash_profile
export JAVA_HOME=/usr/jdk1.8.0_171
export HADOOP_HOME=/usr/hadoop-2.10.0
export PATH=$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH
7.同步时钟
[root@bigdata01 ~]# yum -y install ntpdate
[root@bigdata01 ~]# ntpdate -u ntp.api.bz
[root@bigdata01 ~]# date // 查看系统时间
8.启动Zookeeper集群
# zookeeper集群需要另外关闭,关闭参考zookeeper集群搭建
-
配置Hadoop相关配置文件
注意: 所有的配置文件中不要添加中文注释,同时所有节点均需要进行配置
-
配置core-site.xml
文件路径:${HADOOP_HOME}/etc/hadoop/core-site.xml
<!-- 对namenode做命名服务 --> <property> <name>fs.defaultFS</name> <value>hdfs://mycluster</value> </property> <!-- 临时文件夹 --> <property> <name>hadoop.tmp.dir</name> <value>/usr/hadoop-2.10.0/hadoop-${user.name}</value> </property> <!-- 垃圾回收 --> <property> <name>fs.trash.interval</name> <value>30</value> </property>
-
配置hdfs-site.xml
文件路径:${HADOOP_HOME}/etc/hadoop/hdfs-site.xml
<!-- 副本个数 --> <property> <name>dfs.replication</name> <value>3</value> </property> <!-- 是否启动自动故障转移 --> <property> <name>dfs.ha.automatic-failover.enabled</name> <value>true</value> </property> <!-- 配置zk集群 --> <property> <name>ha.zookeeper.quorum</name> <value>bigdata01:2181,bigdata02:2181,bigdata03:2181</value> </property> <!-- 解析core-site.xml中的mycluster配置--> <property> <name>dfs.nameservices</name> <value>mycluster</value> </property> <property> <name>dfs.ha.namenodes.mycluster</name> <value>nn1,nn2</value> </property> <property> <name>dfs.namenode.rpc-address.mycluster.nn1</name> <value>bigdata01:9000</value> </property> <property> <name>dfs.namenode.rpc-address.mycluster.nn2</name> <value>bigdata02:9000</value> </property> <property> <name>dfs.namenode.shared.edits.dir</name> <value>qjournal://bigdata01:8485;bigdata02:8485;bigdata03:8485/mycluster</value> </property> <property> <name>dfs.client.failover.proxy.provider.mycluster</name> <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value> </property> <property> <name>dfs.ha.fencing.methods</name> <value>sshfence</value> </property> <property> <name>dfs.ha.fencing.ssh.private-key-files</name> <value>/root/.ssh/id_rsa</value> </property>
-
编辑slaves
文件路径: ${HADOOP_HOME}/etc/hadoop/slaves
[root@bigdataX hadoop]# vim slaves bigdata01 bigdata02 bigdata03
-
修改 hadoop-env.sh
文件路径:${HADOOP_HOME}/etc/hadoop/hadoop-env.sh# 修改JAVA_HOME [root@bigdataX hadoop]# vim hadoop-env.sh export JAVA_HOME=/usr/jdk1.8.0_171
-
集群启动
- 启动之前需要启动zk集群
- 启动HDFS服务(进行初始化)
[root@bigdataX ~]# rm -rf ${HADOOP_HOME}/hadoop-root/* // 存在就删除,不存在可以不执行 [root@bigdata01 ~]# hadoop-daemons.sh start journalnode // 任意节点启动均可 [root@bigdata01 ~]# hdfs namenode -format // 进行初始化 [root@bigdata01 ~]# hadoop-daemon.sh start namenode // 启动namenode [root@bigdata02 ~]# hdfs namenode -bootstrapStandby // 下载namenode的元数据 [root@bigdata02 ~]# hadoop-daemon.sh start namenode // 启动namenode,作为01的namenode的从节点 [root@bigdata01 ~]# hdfs zkfc -formatZK // 在任意的01|02节点进行格式化即可 [root@bigdata01 ~]# hadoop-daemon.sh start zkfc // 启动 ZooKeeperFailoverController [root@bigdata02 ~]# hadoop-daemon.sh start zkfc // 启动 ZooKeeperFailoverController [root@bigdata0X ~]# hadoop-daemon.sh start datanode // 启动datanode节点 # 查看每台节点上的进程 [root@bigdata01 ~]# jps 4050 DFSZKFailoverController # ZooKeeperFailoverController 1813 QuorumPeerMain # zookeeper进程 4214 Jps 3735 JournalNode # 日志文件,JournalNode的作用是存放EditLog 3882 NameNode # HDFS的namenode 4142 DataNode # HDFS的dataname [root@bigdata02 ~]# jps 5746 DFSZKFailoverController 3830 QuorumPeerMain 5623 NameNode 5831 DataNode 5866 Jps 5485 JournalNode [root@bigdata03 hadoop-2.10.0]# jps 2891 QuorumPeerMain 4444 DataNode 4365 JournalNode 4526 Jps
- 页面访问与常见命令
web页面访问:ip:50070
- 日常维护命令
# 关闭集群 [root@bigdata01 ~]# stop-dfs.sh Stopping namenodes on [bigdata01 bigdata02] bigdata01: stopping namenode bigdata02: stopping namenode bigdata01: stopping datanode bigdata02: stopping datanode bigdata03: stopping datanode Stopping journal nodes [bigdata01 bigdata02 bigdata03] bigdata02: stopping journalnode bigdata01: stopping journalnode bigdata03: stopping journalnode Stopping ZK Failover Controllers on NN hosts [bigdata01 bigdata02] bigdata02: stopping zkfc bigdata01: stopping zkfc # 启动集群前首先确保zk集群启动,然后启动集群 [root@bigdata01 ~]# start-dfs.sh Starting namenodes on [bigdata01 bigdata02] bigdata01: starting namenode, logging to /usr/hadoop-2.10.0/logs/hadoop-root-namenode-bigdata01.out bigdata02: starting namenode, logging to /usr/hadoop-2.10.0/logs/hadoop-root-namenode-bigdata02.out bigdata03: starting datanode, logging to /usr/hadoop-2.10.0/logs/hadoop-root-datanode-bigdata03.out bigdata01: starting datanode, logging to /usr/hadoop-2.10.0/logs/hadoop-root-datanode-bigdata01.out bigdata02: starting datanode, logging to /usr/hadoop-2.10.0/logs/hadoop-root-datanode-bigdata02.out Starting journal nodes [bigdata01 bigdata02 bigdata03] bigdata01: starting journalnode, logging to /usr/hadoop-2.10.0/logs/hadoop-root-journalnode-bigdata01.out bigdata03: starting journalnode, logging to /usr/hadoop-2.10.0/logs/hadoop-root-journalnode-bigdata03.out bigdata02: starting journalnode, logging to /usr/hadoop-2.10.0/logs/hadoop-root-journalnode-bigdata02.out Starting ZK Failover Controllers on NN hosts [bigdata01 bigdata02] bigdata02: starting zkfc, logging to /usr/hadoop-2.10.0/logs/hadoop-root-zkfc-bigdata02.out bigdata01: starting zkfc, logging to /usr/hadoop-2.10.0/logs/hadoop-root-zkfc-bigdata01.out
Zookeeper集群搭建
以下配置需要在每一个节点都进行配置:
- 修改zoo.cfg配置文件(复制zoo_sample.cfg文件,并更名为zoo.cfg)
tickTime=2000 initLimit=10 syncLimit=5 dataDir=/usr/zookeeper-3.4.14/data clientPort=2181 server.1=bigdata01:2887:3887 server.2=bigdata02:2887:3887 server.3=bigdata03:2887:3887
注意:
上述为ZK节点列表,格式:server.sid = hostname:followingPort:electionPort,如果采用UDP方式,electionPort可以不用设置。
指出了不同的zk的服务器的自身的标示,作为集群中一部分及其应该知道集群中其他的机器,在datadir目录中创建一个文件名为myid的文件,这个文件仅仅含有一行内容,制定的是自身的id值。比如服务器的id是1就在这个文件写1,第一个port是保持和主机通信,第二个port是做选举的。
- 创建data文件夹,作为dataDir
[root@bigdata01 zookeeper-3.4.14]# mkdir data # 在data文件夹中创建 myid文件,在在文件中写入对应节点 id 即可 [root@bigdata01 data]# vim myid [root@bigdata01 data]# cat myid 1
- 配置完成启动即可
# 分别启动节点的zookeeper服务 [root@bigdataX zookeeper-3.4.14]# bin/zkServer.sh start conf/zoo.cfg # 分别关闭节点的zookeeper服务 [root@bigdataX zookeeper-3.4.14]# bin/zkServer.sh stop conf/zoo.cfg # 查看zk服务节点状态 [root@bigdata01 zookeeper-3.4.14]# bin/zkServer.sh status ZooKeeper JMX enabled by default Using config: /usr/zookeeper-3.4.14/bin/../conf/zoo.cfg Mode: leader [root@bigdata03 zookeeper-3.4.14]# jps 2962 Jps 2891 QuorumPeerMai # zookeeper进程
YARN高可用
- 配置yarn-site.xml
文件路径:${HADOOP_HOME}/etc/hadoop/yarn-site.xml<property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <!--Resource Manager--> <property> <name>yarn.resourcemanager.ha.enabled</name> <value>true</value> </property> <property> <name>yarn.resourcemanager.zk-address</name> <value>bigdata01:2181,bigdata02:2181,bigdata03:2181</value> </property> <property> <name>yarn.resourcemanager.cluster-id</name> <value>rmcluster01</value> </property> <property> <name>yarn.resourcemanager.ha.rm-ids</name> <value>rm1,rm2</value> </property> <property> <name>yarn.resourcemanager.hostname.rm1</name> <value>bigdata02</value> </property> <property> <name>yarn.resourcemanager.hostname.rm2</name> <value>bigdata03</value> </property>
- 配置mapred-site.xml
文件路径:${HADOOP_HOME}/etc/hadoop/mapred-site.xml<property> <name>mapreduce.framework.name</name> <value>yarn</value> </property>
- 启动与关闭
# 启动 [[root@bigdata02 ~]# yarn-daemon.sh start resourcemanager [[root@bigdata03 ~]# yarn-daemon.sh start resourcemanager [[root@bigdata0X ~]# yarn-daemon.sh start nodemanager # 关闭 [[root@bigdata02 ~]# yarn-daemon.sh stop resourcemanager [[root@bigdata03 ~]# yarn-daemon.sh stop resourcemanager [[root@bigdata0X ~]# yarn-daemon.sh stop nodemanager
Hbase集群搭建
- 必须安装Hadoop环境,并且配置
HADOOP_HOME
,Hbase会自动读取HDFS - 启动ZK集群
- 上传安装包
- 拷贝到需要安装的节点,进行解压
# 拷贝文件
[root@bigdata01 ~]# scp hbase-2.2.4-bin.tar.gz bigdata0X:/root
# 解压文件
[root@bigdataX ~]# tar -zxvf hbase-2.2.4-bin.tar.gz -C /usr
# 添加环境变量
[root@bigdataX ~]# vim .bash_profile
export HBASE_HOME=/usr/hbase-2.2.4
export HBASE_MANAGES_ZK=false
export PATH=$PATH:$HBASE_HOME/bin
# 环境变量生效
[root@bigdataX ~]# source .bash_profile
-
修改hbase-site.xml
/usr/hbase-2.2.4/conf/hbase-site.xml
<property>
<name>hbase.rootdir</name>
<value>hdfs://mycluster/hbase</value>
</property>
<property>
<!-- zk集群,中文注释记得删除 -->
<name>hbase.zookeeper.quorum</name>
<value>bigdata01,bigdata02,bigdata03</value>
</property>
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2181</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property
<property>
<name>hbase.unsafe.stream.capability.enforce</name>
<value>false</value>
</property>
- 修改regionservers
[root@bigdataX ~]# vim /home/hbase/hbase-1.2.4/conf/regionservers
bigdata01
bigdata02
bigdata03
- 启动&停止
# 启动master
[root@bigdataX ~]# hbase-daemon.sh start master
# 启动regionserver
[root@bigdataX ~]# hbase-daemon.sh start regionserver
# 停止master
[root@bigdataX ~]# hbase-daemon.sh stop master
# 停止regionserver
[root@bigdataX ~]# hbase-daemon.sh stop regionserver
# 日常维护
# 启动
[root@bigdata0X ~]# strat-hbase.sh
# 停止
[root@bigdata0X ~]# stop-hbase.sh