Hadoop的HA集群搭建详细步骤

最新推荐文章于 2024-08-04 13:23:56 发布

卡列宁的微笑66

最新推荐文章于 2024-08-04 13:23:56 发布

阅读量539

点赞数 1

分类专栏： Hadoop

本文链接：https://blog.csdn.net/fczheng/article/details/97899846

版权

本文详细介绍了如何搭建Hadoop的高可用(HA)集群，包括环境准备、集群规划、Linux配置、Hadoop安装、Zookeeper集群安装以及启动Hadoop-HA集群的每个步骤。内容涵盖主机名更改、防火墙关闭、SSH无密登录、JDK安装、Hadoop配置文件分发等关键操作。

摘要由CSDN通过智能技术生成

一、环境准备

1. 修改IP

2. 修改主机名及主机名和IP地址的映射

3. 关闭防火墙

4. ssh免密登录

5. 安装JDK，配置环境变量等

二、集群规划

表2-1 hadoop集群规划
hadoop112	hadoop113	hadoop114
NameNode	NameNode
JournalNode	JournalNode	JournalNode
DataNode	DataNode	DataNode
Zookeeper	Zookeeper	Zookeeper
ResourceManager	ResourceManager
NodeManager	NodeManager	NodeManager

三、Linux环境准备

1、更改主机名

（注：此操作分别在hadoop112,hadoop113,hadoop114上操作）


    [hadoop@hadoop112 ~]$ su root
    [root@hadoop112 hadoop]# vi /etc/sysconfig/network

输入以下内容：

NETWORKING=yes
HOSTNAME=hadoop112

确认主机名生效命令：

[root@hadoop112 hadoop]# hostname hadoop112

是否修改成功命令如下，在操作之前需要关闭当前终端，重新打开一个终端：

[root@hadoop112 ~]# hostname
hadoop112
[root@hadoop112 ~]#

2、关闭防火墙

（注：此操作分别在hadoop112,hadoop113,hadoop114上操作）

关闭防火墙：

[root@hadoop112 hadoop]# service iptables stop

查看防火墙状态（如下图）：

开机禁用防火墙：

[root@hadoop112 hadoop]# chkconfig iptables off

3、配置hosts列表

（注：此操作分别在hadoop112,hadoop113,hadoop114上操作）

 	
 [root@hadoop112 hadoop]# su root
 [root@hadoop112 hadoop]# vi /etc/hosts

输入以下内容：

    192.168.1.112 hadoop112
    192.168.1.113 hadoop113
    192.168.1.114 hadoop114

4、SSH无密登录配置

a、生成公钥和私钥（注：此操作分别在hadoop112,hadoop113,hadoop114上操作）：

然后敲（三个回车），就会生成两个文件id_rsa（私钥）、id_rsa.pub（公钥）
显示如下：

[hadoop@hadoop112 .ssh]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/hadoop/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/hadoop/.ssh/id_rsa.
Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.
The key fingerprint is:
2b:c1:10:22:cc:0f:9d:c9:f1:57:28:be:c5:71:17:1b hadoop@hadoop100
The key's randomart image is:
+--[ RSA 2048]----+
|+ +.=   .. E.    |
| = *.o o.. .o    |
|  o o.o.o ..     |
|   . +.o         |
|      = S        |
|     . . .       |
|      . .        |
|       .         |
|                 |
+-----------------+

b、将公钥拷贝到要免密登录的目标机器上：

[hadoop@hadoop112 .ssh]$ ssh-copy-id hadoop113
[hadoop@hadoop112 .ssh]$ ssh-copy-id hadoop114
[hadoop@hadoop113 .ssh]$ ssh-copy-id hadoop112
[hadoop@hadoop113 .ssh]$ ssh-copy-id hadoop114
[hadoop@hadoop114 .ssh]$ ssh-copy-id hadoop112
[hadoop@hadoop114 .ssh]$ ssh-copy-id hadoop113

注意：ssh访问自己也需要输入密码，所以我们需要将公钥也分别拷贝给自己

[hadoop@hadoop112 .ssh]$ ssh-copy-id hadoop112
[hadoop@hadoop113 .ssh]$ ssh-copy-id hadoop113
[hadoop@hadoop114 .ssh]$ ssh-copy-id hadoop113

c、测试免秘钥登录是否成功

[hadoop@hadoop112.ssh]$ ssh hadoop113
Last login: Mon Jul 22 20:22:44 2019 from 192.168.1.10
[hadoop@hadoop113~]$ exit
logout
Connection to hadoop112 closed.
[hadoop@hadoop112 .ss