Hadoop、Zookeeper、kafka集群启停脚本+文件分发脚本+状态查看脚本
说明:
以下脚本是基于三台服务器,主机名分别为hadoop102、hadoop103、hadoop104
且已配置好SSH服务器免密登录
写完后,勿忘为脚本添加执行权限: chmod 777 xxx.sh 或 chmod +x xxx.sh
[spring@hadoop102 ~] 说明:本机用户名为spring,@后为主机名,脚本编辑需按实际修改即可(主要根据自我实际修改用户名、主机名以及安装路径)。
脚本文件均放在以下目录:
[spring@hadoop102 bin]$ pwd
/home/spring/bin
一、Hadoop集群启停脚本
1. 在~/bin/目录下创建mycluster.sh
[spring@hadoop102 ~]$ cd bin
[spring@hadoop102 bin]$ touch mycluster.sh
[spring@hadoop102 bin]$ vim mycluster.sh
2.集群编写
#!/bin/bash
if [ $# -lt 1 ]
then
echo "No Args Input Error!!!!"
exit
fi
case $1 in
"start")
echo "=============== start hdfs ==============="
ssh hadoop102 /opt/module/hadoop-3.1.3/sbin/start-dfs.sh
echo "=============== start yarn ==============="
ssh hadoop103 /opt/module/hadoop-3.1.3/sbin/start-yarn.sh
;;
"stop")
echo "=============== stop yarn ==============="
ssh hadoop103 /opt/module/hadoop-3.1.3/sbin/stop-yarn.sh
echo "=============== stop hdfs ==============="
ssh hadoop102 /opt/module/hadoop-3.1.3/sbin/stop-dfs.sh
;;
*)
echo "Input Args Error!!!!"
;;
esac
3.脚本使用
mycluster.sh start
mycluster.sh stop
二、Zookeeper启停脚本:
1. 在~/bin/目录下创建zk.sh
[spring@hadoop102 ~]$ cd bin
[spring@hadoop102 bin]$ touch zk.sh
[spring@hadoop102 bin]$ vim zk.sh
2.脚本编写
#/bin/bash
if [ $# -lt 1 ]
then
echo "No Args Input..."
exit;
fi
for i in hadoop102 hadoop103 hadoop104
do
case $1 in
"start")
echo "=============== START $i ZOOKEEPER ==============="
ssh $i /opt/module/zookeeper-3.5.7/bin/zkServer.sh start
;;
"status")
echo "=============== STATUS $i ZOOKEEPER ==============="
ssh $i /opt/module/zookeeper-3.5.7/bin/zkServer.sh status
;;
"stop")
echo "=============== STOP $i ZOOKEEPER ==============="
ssh $i /opt/module/zookeeper-3.5.7/bin/zkServer.sh stop
;;
*)
echo "Input Args Error..."
;;
esac
done
3.脚本使用:
启动: zk.sh start
关闭: zk.sh stop
查看状态: zk.sh status
三、kafka集群启停脚本:
1. 在~/bin/目录下创建kafka-start.sh (以及kafka-stop.sh)
[spring@hadoop102 ~]$ cd bin
[spring@hadoop102 bin]$ touch kafka-start.sh
[spring@hadoop102 bin]$ vim kafka-start.sh
2. 群起脚本(两种方法)
(1) kafka群起脚本(法一):
#!/bin/bash
for((i=102;i<105;i++))
do
echo ----------启动kafka hadoop$i ----------
ssh spring@hadoop$i 'source /etc/profile && /opt/module/kafka_2.11-2.4.1/bin/kafka-server-start.sh -daemon /opt/module/kafka_2.11-2.4.1/config/server.properties'
done
(2) kafka群起脚本(法二):
#kafka群起脚本
#!/bin/bash
#定义linux集群
brokers="hadoop102 hadoop103 hadoop104"
#定义kafka目录
KAFKA_HOME="/opt/module/kafka_2.11-2.4.1"
KAFKA_NAME="Kafka"
echo "============== 开始启动Kafka集群 ==============="
for broker in $brokers
do
ssh ${broker} -C "source /etc/profile; ${KAFKA_HOME}/bin/kafka-server-start.sh -daemon ${KAFKA_HOME}/config/server.properties"
if [[ $? -eq 0 ]]; then
echo "INFO:======= [${broker}] Start successfully =========="
fi
done
echo "============== Kafka集群启动成功! ================"
3. 群停脚本(两种方法)
(1) kafka集群停止脚本(法一):
#!/bin/bash
for((i=102;i<105;i++))
do
echo ----------hadoop$i----------
ssh spring@hadoop$i 'source /etc/profile && /opt/module/kafka_2.11-2.4.1/bin/kafka-server-stop.sh'
done
(2) kafka集群停止脚本(法二):
#!/bin/bash
#Kafka集群停止脚本
#定义linux集群
brokers="hadoop102 hadoop103 hadoop104"
#定义kafka目录
KAFKA_HOME="/opt/module/kafka_2.11-2.4.1"
KAFKA_NAME="Kafka"
echo "INFO : ============ 开始停止Kafka集群 ============..."
for broker in $brokers
do
echo "INFO : Shut down ${KAFKA_NAME} on ${broker} ..."
ssh ${broker} "source /etc/profile; ${KAFKA_HOME}/bin/kafka-server-stop.sh"
if [[ $? -ne 0 ]]; then
echo "INFO : Shut down ${KAFKA_NAME} on ${broker} is down"
fi
done
echo "INFO : ============ Kafka集群已关闭! ===========..."
四、状态查看(jps)
1. 在~/bin/目录下创建myjps.sh
[spring@hadoop102 ~]$ cd bin
[spring@hadoop102 bin]$ touch myjps.sh
[spring@hadoop102 bin]$ vim myjps.sh
2. 编写脚本
#!/bin/bash
for i in hadoop102 hadoop103 hadoop104
do
echo "=============== $i JPS ==============="
ssh $i /opt/module/jdk1.8.0_212/bin/jps
done
注:jps是JDK中的命令,不是Linux命令。不安装JDK不能使用jps命令
3.脚本使用
myjps.sh
五、分发脚本
1. 在~/bin/目录下创建xsync.sh
[spring@hadoop102 ~]$ cd bin
[spring@hadoop102 bin]$ touch xsync.sh
[spring@hadoop102 bin]$ vim xsync.sh
2.脚本编写
#!/bin/bash
#1. 判断参数个数
if [ $# -lt 1 ]
then
echo Not Enough Arguement!
exit;
fi
#2. 遍历集群所有机器
for host in hadoop102 hadoop103 hadoop104
do
echo ==================== $host ====================
#3. 遍历所有目录,挨个发送
for file in $@
do
#4 判断文件是否存在
if [ -e $file ]
then
#5. 获取父目录
pdir=$(cd -P $(dirname $file); pwd)
#6. 获取当前文件的名称
fname=$(basename $file)
ssh $host "mkdir -p $pdir"
rsync -av $pdir/$fname $host:$pdir
else
echo $file does not exists!
fi
done
done
3.脚本使用:
xsync.sh 文件