Linux集群实用脚本

现有三台虚拟机，例如：node2、node3、node4

三台虚拟机配置了免密登录，安装了hadoop等软件。

相关集群实用脚本命令

统一执行jps命令

jpsall

#!/bin/bash
​
for host in node2 node3 node4
do
        echo =============== $host ===============
        ssh $host jps 
done
​

统一执行命令

集群统一执行命令，执行的命令通过参数的执行传入，更具有灵活性。

same.sh

#!/bin/bash
​
# 1.获取参数个数，小于1个参数报错
if [ $# -lt 1 ]
then
    echo "No Args command Input..."
    exit ;
fi
​
# 2.获取当前机器的路径
currDir=$pwd
​
# 3.ssh到每一台机器，切换到执行脚本机器的当前目录并执行相应命令，这里执行的命令只支持3个参数，可自己根据实际情况扩展，一般用于查看路径或文件内容
for host in node2 node3 node4
do
        echo =============== $host ===============
        ssh $host "cd $currDir;$1 $2 $3;" 
done
​

例如：统一使用ls命令查看三台机器的/home目录，命令如下

统一关机脚本

guanji.sh

#!/bin/bash
​
for host in node4 node3 node2
do
        echo =============== $host ===============
        ssh $host sudo init 0; 
done
​

hadoop启动/停止脚本

#!/bin/bash

if [ $# -lt 1 ]
then
    echo "No Args Input..."
    exit ;
fi

case $1 in
"start")
        echo " =================== 启动 hadoop集群 ==================="

        echo " --------------- 启动 hdfs ---------------"
        ssh node2 "/home/hadoop/soft/hadoop-3.1.3/sbin/start-dfs.sh"
        echo " --------------- 启动 yarn ---------------"
        ssh node3 "/home/hadoop/soft/hadoop-3.1.3/sbin/start-yarn.sh"
        echo " --------------- 启动 historyserver ---------------"
        ssh node2 "/home/hadoop/soft/hadoop-3.1.3/bin/mapred --daemon start historyserver"
;;
"stop")
        echo " =================== 关闭 hadoop集群 ==================="

        echo " --------------- 关闭 historyserver ---------------"
        ssh node2 "/home/hadoop/soft/hadoop-3.1.3/bin/mapred --daemon stop historyserver"
        echo " --------------- 关闭 yarn ---------------"
        ssh node3 "/home/hadoop/soft/hadoop-3.1.3/sbin/stop-yarn.sh"
        echo " --------------- 关闭 hdfs ---------------"
        ssh node2 "/home/hadoop/soft/hadoop-3.1.3/sbin/stop-dfs.sh"
;;
*)
    echo "Input Args Error..."
;;
esac

zk启动/停止脚本

zk.sh

#!/bin/bash

case $1 in
"start"){
	for i in node2 node3 node4
	do
        echo ---------- zookeeper $i 启动 ------------
		ssh $i "/home/hadoop/soft/apache-zookeeper-3.7.1-bin/bin/zkServer.sh start"
	done
};;
"stop"){
	for i in node2 node3 node4
	do
        echo ---------- zookeeper $i 停止 ------------    
		ssh $i "/home/hadoop/soft/apache-zookeeper-3.7.1-bin/bin/zkServer.sh stop"
	done
};;
"status"){
	for i in node2 node3 node4
	do
        echo ---------- zookeeper $i 状态 ------------    
		ssh $i "/home/hadoop/soft/apache-zookeeper-3.7.1-bin/bin/zkServer.sh status"
	done
};;
esac

kafka启动/停止脚本

kf.sh

#! /bin/bash

case $1 in
"start"){
    for i in node2 node3 node4
    do
        echo " --------启动 $i Kafka-------"
        ssh $i "/home/hadoop/soft/kafka_2.12-3.3.1/bin/kafka-server-start.sh -daemon /home/hadoop/soft/kafka_2.12-3.3.1/config/server.properties"
    done
};;
"stop"){
    for i in node2 node3 node4
    do
        echo " --------停止 $i Kafka-------"
        ssh $i "/home/hadoop/soft/kafka_2.12-3.3.1/bin/kafka-server-stop.sh stop"
    done
};;
esac

 

maxwell启动/停止脚本

mxw.sh

#!/bin/bash

MAXWELL_HOME=/home/hadoop/soft/maxwell-1.29.2

status_maxwell(){
    result=`ps -ef | grep com.zendesk.maxwell.Maxwell | grep -v grep | wc -l`
    return $result
}

start_maxwell(){
    status_maxwell
    if [[ $? -lt 1 ]]; then
        echo "启动Maxwell"
        $MAXWELL_HOME/bin/maxwell --config $MAXWELL_HOME/config.properties --daemon
    else
        echo "Maxwell正在运行"
    fi
}

stop_maxwell(){
    status_maxwell
    if [[ $? -gt 0 ]]; then
        echo "停止Maxwell"
        ps -ef | grep com.zendesk.maxwell.Maxwell | grep -v grep | awk '{print $2}' | xargs kill -9
    else
        echo "Maxwell未在运行"
    fi
}


case $1 in
    start )
        start_maxwell
    ;;
    stop )
        stop_maxwell
    ;;
    restart )
       stop_maxwell
       start_maxwell
    ;;
esac

 

一键启动/停止zk、kafka、maxwell脚本

zkm.sh

#! /bin/bash
case $1 in
"start"){
    zk.sh start
	kf.sh start
	ssh node3 "/home/hadoop/bin/mxw.sh start"
};;
"stop"){
	ssh node3 "/home/hadoop/bin/mxw.sh stop"
    kf.sh stop
    zk.sh stop
};;
esac

 

注意：

1.脚本名称可以自定义，但不要与原有命令重复；

2.需要给脚本添加可执行权限；

3.为了能更加方便使用脚本，将脚本所在目录添加到环境变量中。

完成！enjoy it！