HA高可用配置

HA （High Available）配置

详见http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html进行参考配置
本指南概述了 HDFS 高可用性 (HA) 功能以及如何使用 Quorum Journal Manager (QJM) 功能配置和管理 HA HDFS 集群。
（一）集群的规划

（二）创建虚拟机

1. 配置网络
   查看子网ip
   

vi /etc/sysconfig/network-scripts/ifcfg-ens33

2. 配置主机名

hostnamectl set-hostname ha-01

其他两个也需在相应窗口配置

hostnamectl set-hostname ha-02
hostnamectl set-hostname ha-03

3. 免密登录
   在secureCRT中send to all sessions窗口输入以下内容

ssh-keygen -t rsa

ssh-copy-id ha-01
ssh-copy-id ha-02
ssh-copy-id ha-03

4. 创建目录
压缩包放在software
hadoop zookeeper解压至servers下

mkdir export
cd export
mkdir data
mkdir servers
mkdir software

5.克隆出两台虚拟机ha-02,ha-03

配置各虚拟机ip地址和主机名之间的映射关系

vi /etc/hosts

添加以下内容
192.168.88.151 ha-01
192.168.88.152 ha-02
192.168.88.153 ha-03
或者

echo '192.168.88.151 ha-01
> 192.168.88.152 ha-02
> 192.168.88.153 ha-03
> ' >>/etc/hosts

分发/etc/hosts

for i in `seq 2 3`
do
scp -r /etc/hosts ha-0$i:/etc/
scp -r /etc/hosts ha-0$i:/etc/
done

Windows下编辑/etc/hosts
192.168.88.151 ha-01
192.168.88.152 ha-02
192.168.88.153 ha-03

安装配置hadoop，jdk、zookeeper环境变量

#Hadoop
export HADOOP_HOME=/export/servers/hadoop-3.3.1
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
#jdk
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.302.b08-0.el8_4.x86_64/jre
export PATH=$PATH:$JAVA_HOME/bin
#zookeeper
export ZOOKEEPER_HOME=/export/servers/apache-zookeeper-3.6.3-bin
export PATH=$PATH:$ZOOKEEPER_HOME/bin

这里jps 为系统自带版本没有jps
需要下载

yum -y  install java-1.8.0-openjdk-devel.x86_64

hadoop

hdfs.site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->



<configuration>
	<!--设置副本的个数 -->
	<property>
		<name>dfs.replication</name>
		<value>2</value>
	</property>
	<!