Hadoop群起脚本及(Zookeeper&kafka)

详细脚本如下

群起Hadoop

#!/bin/bash
echo "=======================正在开启集群服务======================"
echo "********************正在开启NameNode(master)节点*******************"
ssh hadoop@master '/opt/module/hadoop-2.7.2/sbin/hadoop-daemon.sh start namenode'

echo "=======================正在开启DataNode节点=================="
for i in hadoop@master hadoop@slave1 hadoop@slave2
do
    ssh $i '/opt/module/hadoop-2.7.2/sbin/hadoop-daemon.sh start datanode'
done

echo "=======================正在开启SeondaryNameNode(slave2)节点=================="
ssh hadoop@slave2 '/opt/module/hadoop-2.7.2/sbin/hadoop-daemon.sh start secondarynamenode'

echo "*******************正在开启ResourceManager(slave1)节点********************"
ssh hadoop@slave1 '/opt/module/hadoop-2.7.2/sbin/yarn-daemon.sh start resourcemanager'

echo "***************正在开启JobHistoryServer(master)****************"
ssh hadoop@master '/opt/module/hadoop-2.7.2/sbin/mr-jobhistory-daemon.sh start historyserver'

echo "=======================正在开启NodeManager节点=================="
for i in hadoop@master hadoop@slave1 hadoop@slave2
do
        ssh $i '/opt/module/hadoop-2.7.2/sbin/yarn-daemon.sh start nodemanager'
done

群关Hadoop

#!/bin/bash
echo "=======================正在开启集群服务======================"
echo "********************正在开启NameNode(master)节点*******************"
ssh hadoop@master '/opt/module/hadoop-2.7.2/sbin/hadoop-daemon.sh stop namenode'

echo "=======================正在开启DataNode节点=================="
for i in hadoop@master hadoop@slave1 hadoop@slave2
do
    ssh $i '/opt/module/hadoop-2.7.2/sbin/hadoop-daemon.sh stop datanode'
done

echo "=======================正在开启SeondaryNameNode(slave2)节点=================="
ssh hadoop@slave2 '/opt/module/hadoop-2.7.2/sbin/hadoop-daemon.sh stop secondarynamenode'

echo "*******************正在开启ResourceManager(slave1)节点********************"
ssh hadoop@slave1 '/opt/module/hadoop-2.7.2/sbin/yarn-daemon.sh stop resourcemanager'

echo "***************正在开启JobHistoryServer(master)****************"
ssh hadoop@master '/opt/module/hadoop-2.7.2/sbin/mr-jobhistory-daemon.sh stop historyserver'

echo "=======================正在开启NodeManager节点=================="
for i in hadoop@master hadoop@slave1 hadoop@slave2
do
        ssh $i '/opt/module/hadoop-2.7.2/sbin/yarn-daemon.sh stop nodemanager'
done

查看所有节点jps:

 #!/bin/bash
    for i in hadoop@master hadoop@slave1 hadoop@slave2
    do
            echo "================      $i        ================"
            ssh $i '/opt/module/jdk1.8.0_144/bin/jps'
    done

群起ZooKeeper

  #!/bin/bash
    for i in hadoop@master hadoop@slave1 hadoop@slave2
    do
            echo "================           $i             ================"
            ssh $i '/opt/module/zookeeper-3.4.10/bin/zkServer.sh start'
    done

群关ZooKeeper

 #!/bin/bash
    for i in hadoop@master hadoop@slave1 hadoop@slave2
    do
            echo "================           $i             ================"
            ssh $i '/opt/module/zookeeper-3.4.10/bin/zkServer.sh stop'
    done

群开、关kafka

#!/bin/bash
case $1 in 
"start"){
    
    for i in master slave1 slave2
    do
    echo "*************** $i ***************"
        ssh $i "/opt/module/kafka/bin/kafka-server-start.sh -daemo /opt/module/kafka/config/server.properties"
    done
};;
"stop"){
    for i in master slave1 slave2
    do
    echo "*************** $i ***************"
        ssh $i "/opt/module/kafka/bin/kafka-server-stop.sh  /opt/module/kafka/config/server.properties"
    done
};;

esac

注意：
1.需要配置/home/hadoop用户目录下的(.bashrc)文件，在其中配置JAVA_HOME，并source 一下/home/hadoop/.bashrc(如果用的是非root用户必须就必须进行配置)
2.在配置完脚本后记得给文件基给予执行权限

[hadoop@master bin]$ chmod 777 zk