大数据学习之使用shell脚本启动、查看、关闭服务

前言：

本篇文章针对于2020秋季学期的复习操作，一是对该学期的巩固，二是让老师知道他的努力没有白费，同时，在此感谢徐老师对我们的精心教导…

介绍

我们知道当我们的集群规模大了以后，并且安装的集群多了以后，启动集群服务非常繁琐，那么，今天的shell脚本将解决这些繁琐的步骤，实现一步到位！！！

实施

前提

建立一个文件夹用于存放shell脚本文件

[root@Mymaster ~]# cd /opt/
[root@Mymaster opt]# mkdir myself-shells
[root@Mymaster opt]# vim /etc/profile.d/bigdata-etc.sh

[root@Mymaster opt]# source /etc/profile.d/bigdata-etc.sh 

编写一个存放各个节点标识的文件

[root@Mymaster opt]# cd myself-shells/
[root@Mymaster myself-shells]# vim nodes
[root@Mymaster myself-shells]# cat nodes
Myslave02
Myslave01
Mymaster

①一键启动zk集群、hadoop集群

新建自己的启动文件

[root@Mymaster myself-shells]# vim myself-start-all.sh

我们知道hadoop是基于zookeeper集群的所以我们要先启动zookeeper集群

#!/bin/sh

# 步骤1：先启动zk分布式集群
nodes=$(cat /opt/myself-shells/nodes)

# echo $nodes
for zk in $nodes
do
    ssh root@$zk 'zkServer.sh start'
done

# 步骤2：然后启动hadoop分布式集群
# ①在Mymaser节点上启动start-dfs.sh
ssh root@Mymaster start-dfs.sh

# ②在Mymaster上启动yarn
ssh root@Mymaster start-yarn.sh

给启动文件添加可执行权限

②一键查看所有节点的进程

[root@Mymaster myself-shells]# vim myself-show-all.sh

#! /bin/sh
nodes=$(cat /opt/myself-shells/nodes)

for node in $nodes
do
    echo "----------------------------- $node ----------------------------"
    ssh root@$node jps
    echo ""
done

③一键关闭zk集群、hadoop集群

[root@Mymaster myself-shells]# vim myself-stop-all.sh

#!/bin/sh

  #①停止hadoop集群
   #   a)在Mymaster节点上执行stop-dfs.sh
ssh Mymaster 'stop-dfs.sh'
    #  b)在Mymater节点上执行stop-yarn.sh
ssh Mymaster 'stop-yarn.sh'
nodes=$(cat /opt/myself-shells/nodes)
#echo $nodes
  #②停止zookeeper集群(通过循环来停止)
for zkk in $nodes
do
    ssh root@$zkk 'zkServer.sh stop'
done

④一键关机所有机器

[root@Mymaster myself-shells]# vim myself-poweroff-all.sh

# 关机
#!/bin/sh
nodes=$(cat /opt/myself-shells/nodes)
#echo $nodes
for shutdown in $nodes
do
        ssh root@$shutdown "shutdown -h now"
done

⑤测试

[root@Mymaster myself-shells]# myself-show-all.sh 
----------------------------- Myslave02 ----------------------------
77050 Jps

----------------------------- Myslave01 ----------------------------
79412 Jps

----------------------------- Mymaster ----------------------------
28297 Jps

[root@Mymaster myself-shells]# myself-start-all.sh 
ZooKeeper JMX enabled by default
Using config: /opt/zk/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
ZooKeeper JMX enabled by default
Using config: /opt/zk/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
ZooKeeper JMX enabled by default
Using config: /opt/zk/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
Starting namenodes on [Mymaster]
Mymaster: starting namenode, logging to /opt/hadoop/logs/hadoop-root-namenode-Mymaster.out
Myslave02: starting datanode, logging to /opt/hadoop/logs/hadoop-root-datanode-Myslave01.out
Mymaster: starting datanode, logging to /opt/hadoop/logs/hadoop-root-datanode-Mymaster.out
Myslave01: starting datanode, logging to /opt/hadoop/logs/hadoop-root-datanode-Myslave01.out
Starting secondary namenodes [Myslave01]
Myslave01: starting secondarynamenode, logging to /opt/hadoop/logs/hadoop-root-secondarynamenode-Myslave01.out
starting yarn daemons
starting resourcemanager, logging to /opt/hadoop/logs/yarn-root-resourcemanager-Mymaster.out
Mymaster: starting nodemanager, logging to /opt/hadoop/logs/yarn-root-nodemanager-Mymaster.out
Myslave02: starting nodemanager, logging to /opt/hadoop/logs/yarn-root-nodemanager-Myslave01.out
Myslave01: starting nodemanager, logging to /opt/hadoop/logs/yarn-root-nodemanager-Myslave01.out
[root@Mymaster myself-shells]# myself-show-all.sh 
----------------------------- Myslave02 ----------------------------
77255 NodeManager
77382 Jps
77146 DataNode
77084 QuorumPeerMain

----------------------------- Myslave01 ----------------------------
79683 NodeManager
79810 Jps
79607 SecondaryNameNode
79446 QuorumPeerMain
79502 DataNode

----------------------------- Mymaster ----------------------------
28945 DataNode
29429 ResourceManager
29673 NodeManager
30057 Jps
28542 QuorumPeerMain
28735 NameNode
[root@Mymaster myself-shells]# myself-stop-all.sh 
Stopping namenodes on [Mymaster]
Mymaster: stopping namenode
Mymaster: stopping datanode
Myslave02: stopping datanode
Myslave01: stopping datanode
Stopping secondary namenodes [Myslave01]
Myslave01: stopping secondarynamenode
stopping yarn daemons
stopping resourcemanager
Mymaster: stopping nodemanager
Myslave02: stopping nodemanager
Myslave01: stopping nodemanager
no proxyserver to stop
ZooKeeper JMX enabled by default
Using config: /opt/zk/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
ZooKeeper JMX enabled by default
Using config: /opt/zk/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
ZooKeeper JMX enabled by default
Using config: /opt/zk/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
[root@Mymaster myself-shells]# myself-show-all.sh 
----------------------------- Myslave02 ----------------------------
77483 Jps

----------------------------- Myslave01 ----------------------------
79949 Jps

----------------------------- Mymaster ----------------------------
33818 Jps

[root@Mymaster myself-shells]# 

那么本次的复习到此就结束了！！！
编写与2021-1-23