1 简介
在大数据集群部署过程中,需要对集群服务启动或者停止,需要同时操作多台机器,可使用集群脚本实现。
2 配置集群hostname
2.1 配置hostname文件
1 服务器hadoop01
[root@localhost ~]# echo hostname1 > /etc/hostname
hostname hadoop01
2 服务器hadoop02
[root@localhost ~]# echo hadoop02 > /etc/hostname
hostname hadoop02
3 服务器hadoop03
[root@localhost ~]# echo hadoop03 > /etc/hostname
hostname hadoop03
4 服务器hadoop04
[root@localhost ~]# echo hadoop04 > /etc/hostname
hostname hadoop04
2.2 配置hosts文件
同步脚本的那台机器上,现在是hadoop01
192.168.85.101 hadoop01
192.168.85.102 hadoop02
192.168.85.103 hadoop03
192.168.85.104 hadoop04
192.168.85.105 hadoop05
3 配置免密登录
3.1 生成rsa密钥
使用命令ssh-keygen
生成rsa密钥,配置信息直接回车即可, 生成的密钥默认在当前用户主目录的.ssh目录下。
密钥文件有两个:
- id_rsa 存放着私钥
- id_rsa.pub 存放着公钥
[root@localhost ~]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:fX0pdntsAyRWTvjuoSG+VFu/vsfFpRypBi2Z5iRY8ok root@localhost.localdomain
The key's randomart image is:
+---[RSA 2048]----+
| .o |
| . . .+ |
| * . *.o . |
| E +.O +oo o|
| S=.+oB.*o|
| .oo*oBo+|
| ...o+ .==|
| .. . . .*|
| .. .+o|
+----[SHA256]-----+
[root@localhost ~]#
3.2 同步公钥到其他服务器
3.2.1 安装rsync
[root@localhost ~]# yum install -y rsync
3.2.1 同步公钥
[root@localhost ~]# cp id_rsa.pub authorized_keys
[root@localhost ~]# rsync authorized_keys root@hadoop02:/root/.ssh/
[root@localhost ~]# rsync authorized_keys root@hadoop03:/root/.ssh/
[root@localhost ~]# rsync authorized_keys root@hadoop04:/root/.ssh/
3.3 免密登录测试
无需输入密码,代表密码登录配置成功
[root@localhost ~]# root@hadoop02
4 脚本使用
用户主目录的bin目录添加
pwd
查看当前目录,用户登录进去的初始目录,一般为家
目录
4.1 kafka集群启动脚本
#!/bin/bash
echo "================ 开始启动所有节点服务 ==========="
echo "================ 正在启动Zookeeper ==========="
for i in auskat@hadoop102 auskat@hadoop103 auskat@hadoop104
do
ssh $i 'source /etc/profile && /opt/module/zookeeper-3.4.10/bin/zkServer.sh start'
done
echo "================ 正在启动Kafka ==========="
for i in auskat@hadoop102 auskat@hadoop103 auskat@hadoop104
do
ssh $i 'source /etc/profile && /opt/module/kafka/bin/kafka-server-start.sh -daemon /opt/module/kafka/config/server.properties'
echo "$i kafka is start"
done
4.2 kafka集群停止脚本
#!/bin/bash
echo "================ 开始启动所有节点服务 ==========="
echo "================ 正在启动Zookeeper ==========="
for i in auskat@hadoop102 auskat@hadoop103 auskat@hadoop104
do
ssh $i 'source /etc/profile && /opt/module/zookeeper-3.4.10/bin/zkServer.sh start'
done
echo "================ 正在启动Kafka ==========="
for i in auskat@hadoop102 auskat@hadoop103 auskat@hadoop104
do
ssh $i 'source /etc/profile && /opt/module/kafka/bin/kafka-server-start.sh -daemon /opt/module/kafka/config/server.properties'
echo "$i kafka is start"
done
4.3 hadoop集群启动脚本
#!/bin/bash
echo "================ 开始启动所有节点服务 ==========="
echo "================ 正在启动Zookeeper ==========="
for i in auskat@hadoop102 auskat@hadoop103 auskat@hadoop104
do
ssh $i '/opt/module/zookeeper-3.4.10/bin/zkServer.sh start'
done
echo "================ 正在启动HDFS ==========="
ssh auskat@hadoop102 '/opt/module/hadoop-2.7.2/sbin/start-dfs.sh'
echo "================ 正在启动YARN ==========="
ssh auskat@hadoop103 '/opt/module/hadoop-2.7.2/sbin/start-yarn.sh'
echo "================ 正在开启JobHistoryServer ==========="
ssh auskat@hadoop104 '/opt/module/hadoop-2.7.2/sbin/mr-jobhistory-daemon.sh start historyserver'
4.4 hadoop集群停止脚本
#!/bin/bash
echo "================ 开始关闭所有节点服务 ==========="
echo "================ 正在关闭Zookeeper ==========="
for i in auskat@hadoop102 auskat@hadoop103 auskat@hadoop104
do
ssh $i '/opt/module/zookeeper-3.4.10/bin/zkServer.sh stop'
done
echo "================ 正在关闭HDFS ==========="
ssh auskat@hadoop102 '/opt/module/hadoop-2.7.2/sbin/stop-dfs.sh'
echo "================ 正在关闭YARN ==========="
ssh auskat@hadoop103 '/opt/module/hadoop-2.7.2/sbin/stop-yarn.sh'
echo "================ 正在关闭JobHistoryServer ==========="
ssh auskat@hadoop104 '/opt/module/hadoop-2.7.2/sbin/mr-jobhistory-daemon.sh stop historyserver'
5 相关信息
- 博文不易,辛苦各位猿友点个关注和赞,感谢