3.5:Zabbix 监控 MySQL
本次实验的目的是:
- 掌握单节点MySQL如何扩展至双节点MySQL主从结构,将现有zabbix的mysql服务器调整为主从异步复制结构;
- 通过Percona Monitoring Plugins for zabbix提供的脚本和监控模板,对MySQL进行监控(了解即可);
- 熟悉了解MySQL主从状态信息,编写脚本,使zabbix对自身的mysql服务器进行监控。
拓扑图
3.5.1:准备 MySQL 主从环境
将Zabbix单节点MySQL扩展为双节点主从结构的步骤如下:
- 确保Master和Slave两台服务器的时间同步一致。
- 安装并启动MySQL-Slave(node200,192.168.1.200)(先只做正常的监听和安全设置,不做主从设置);
- 开启MySQL-Master的binlog(同时必须设置server-id),并对MySQL-Maste进行全量备份(备份时指定–flush-logs重新滚动二进制日志,以方便之后对MySQL-Slave进行恢复binlog的操作);
- MySQL-Slave导入MySQL-Master的全量备份和备份时滚动生成的binlog(这时拷贝binlog时要记录其中最后一个end_log_pos指向的位置,这个位置就是之后为MySQL-Slave指定Master时备份开始的position);
- MySQL-Master的主从相关配置(设置server-id、开启binlog前边已做,这里需要授权主从复制的用户);
- MySQL-Slave的主从相关配置(设置server-id,开启relay-log和read-only,指定Master并开启IO和SQL线程);
- 验证主从结构。
3.5.1.1:MySQL-Slave 部署 MySQL
安装和MySQL-Master相同版本的MySQL,进行初始化安全设置,配置监听:
root@zabbix-mysql-backup:~# apt update
root@zabbix-mysql-backup:~# apt install mysql-server mysql-client
root@zabbix-mysql-backup:~# mysql_secure_installation
root@zabbix-mysql-backup:~# vim /etc/mysql/mysql.conf.d/mysqld.cnf
bind-address = 0.0.0.0
root@zabbix-mysql-backup:~# systemctl restart mysql
3.5.1.2:MySQL-Master 备份数据库
3.5.1.2.1:备份前确认开启 binlog
前边只是测试,所以MySQL并没有开启binlog。
因为之后Master-Slave需要导入全量备份之后生成的binlog,所以要先开启binlog同时将binlog的刷写机制也设置一下:
root@zabbix-mysql:~# vim /etc/mysql/mysql.conf.d/mysqld.cnf
server-id = 100
log_bin = /var/log/mysql/mysql-bin.log
sync_binlog = 1
root@zabbix-mysql:~# systemctl restart mysql
3.5.1.2.2:全量热备
全量热备,同时滚动生成新的binlog:
root@zabbix-mysql:~# mysqldump -uroot -p -hlocalhost --all-databases --single-transaction -R -E --master-data=2 --flush-logs > /root/alldb-$(date +%F).sql
Enter password:
root@zabbix-mysql:~# ll alldb-2021-04-16.sql
-rw-r--r-- 1 root root 17849321 Apr 16 20:02 alldb-2021-04-16.sql
root@zabbix-mysql:~# ll /var/log/mysql/mysql-bin.*
-rw-r----- 1 mysql mysql 31667 Apr 16 19:04 /var/log/mysql/mysql-bin.000001
-rw-r----- 1 mysql mysql 831308 Apr 16 19:32 /var/log/mysql/mysql-bin.000002
-rw-r----- 1 mysql mysql 938409 Apr 16 20:02 /var/log/mysql/mysql-bin.000003
-rw-r----- 1 mysql mysql 17272 Apr 16 20:03 /var/log/mysql/mysql-bin.000004
-rw-r----- 1 mysql mysql 128 Apr 16 20:02 /var/log/mysql/mysql-bin.index
3.5.1.3:MySQL-Slave 导入备份和 binlog
3.5.1.3.1:拷贝并导入备份
root@zabbix-mysql-backup:~# scp 192.168.1.100:/root/alldb-2021-04-16.sql /root/
root@zabbix-mysql-backup:~# mysql -uroot -p -hlocalhost < alldb-2021-04-16.sql
3.5.1.3.2:拷贝 binlog
root@zabbix-mysql-backup:~# scp 192.168.1.100:/var/log/mysql/mysql-bin.000004 /root/
3.5.1.3.3:查看 binlog 中最后一个 position
从最后一个 end_log_pos
可以看出,主从复制的开始位置就是65463,记录下来。
root@zabbix-mysql-backup:~# mysqlbinlog mysql-bin.000004 | tail
MtHr
'/*!*/;
# at 65432
#210416 20:05:03 server id 100 end_log_pos 65463 CRC32 0xd3f664f7 Xid = 35531
COMMIT/*!*/;
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# End of log file
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
3.5.1.3.4:导入 binlog
读取binlog的内容并将其写入zabbix数据库:
root@zabbix-mysql-backup:~# mysqlbinlog mysql-bin.000004 | mysql -uroot -p -hlocalhost zabbix
3.5.1.4:主从配置 - MySQL-Master
创建主从复制时使用的用户,这里是repl,并对其进行主从复制需要的相关授权:
mysql> GRANT REPLICATION SLAVE,REPLICATION CLIENT ON *.* TO 'repl'@'192.168.1.%' IDENTIFIED BY '123456';
mysql> FLUSH PRIVILEGES;
3.5.1.5:主从配置 - MySQL-Slave
设置MySQL-Slave主从复制需要的相关配置项:
root@zabbix-mysql-backup:~# vim /etc/mysql/mysql.conf.d/mysqld.cnf
server-id = 200
relay_log = /var/log/mysql/mysql-relay.log
read_only = ON
sync_master_info = 1
sync_relay_log_info = 1
重启MySQL:
root@zabbix-mysql-backup:~# systemctl restart mysql
为MySQL-Slave指定master,并开启IO和SQL线程:
指定开始同步的位置为拷贝过来的binlog中最后一个
end_log_pos
指向的位置65463。
mysql> CHANGE MASTER TO MASTER_HOST='192.168.1.100',MASTER_USER='repl',MASTER_PASSWORD='123456',MASTER_LOG_FILE='mysql-bin.000004',MASTER_LOG_POS=65463;
mysql> START SLAVE;
3.5.1.6:验证主从结构
MySQL-Slave节点查看主从同步状态:
IO和SQL线程的运行状态为双yes,并且因为zabbix一直在运行,Master的数据是一直在写入的,所以Exec_Master_Log_Pos每次刷新都在变化,就说明主从复制已经正常了。
mysql> SHOW SLAVE STATUS\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.1.100
Master_User: repl
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000004
Read_Master_Log_Pos: 339727
Relay_Log_File: mysql-relay.000002
Relay_Log_Pos: 274584
Relay_Master_Log_File: mysql-bin.000004
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 339727
Relay_Log_Space: 274787
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 100
Master_UUID: b5940e14-7bed-11eb-8e0f-000c298c58ca
Master_Info_File: /var/lib/mysql/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set:
Auto_Position: 0
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)
3.5.2:通过 Percona Monitoring Plugin 监控 MySQL
Percona提供了一个用于zabbix监控MySQL的插件,并有相应的监控模板。
但由于这个插件更新到1.1.8就停更了,最多支持到Unbuntu17.10,正常执行脚本前需要将Unbuntu1804的php版本降级到5.6。
《Percona Monitoring Plugins for Zabbix》文档:https://www.percona.com/doc/percona-monitoring-plugins/LATEST/zabbix/index.html
Percona Monitoring Plugins 下载:https://www.percona.com/downloads/percona-monitoring-plugins/LATEST/
下面在MySQL-Master上部署插件,对其MySQL运行状态进行监控。
3.5.2.1:MySQL-Master 安装 zabbix-agent
apt安装:
root@zabbix-mysql:~# wget https://repo.zabbix.com/zabbix/4.0/ubuntu/pool/main/z/zabbix-release/zabbix-release_4.0-3+bionic_all.deb
root@zabbix-mysql:~# dpkg -i zabbix-release_4.0-3+bionic_all.deb
root@zabbix-mysql:~# sed -i 's@repo.zabbix.com/zabbix/4.0@mirrors.aliyun.com/zabbix/zabbix/4.0@g' /etc/apt/sources.list.d/zabbix.list
root@zabbix-mysql:~# apt update
root@zabbix-mysql:~# apt install zabbix-agent
配置:
root@zabbix-mysql:~# grep '^[a-Z]' /etc/zabbix/zabbix_agentd.conf
PidFile=/var/run/zabbix/zabbix_agentd.pid
LogFile=/var/log/zabbix/zabbix_agentd.log
LogFileSize=0
EnableRemoteCommands=1
Server=192.168.1.101
ListenPort=10050
ServerActive=192.168.1.101
Hostname=192.168.1.100
Include=/etc/zabbix/zabbix_agentd.d/*.conf
重启:
root@zabbix-mysql:~# systemctl enable zabbix-agent && systemctl restart zabbix-agent
验证:
root@zabbix-mysql:~# ss -tnlp | grep 10050
3.5.2.2:MySQL-Master 部署 Percona Monitoring Plugins
3.5.2.2.1:下载
下载地址就是上面列出的https://www.percona.com/downloads/percona-monitoring-plugins/LATEST/
选择最后一个版本1.1.8,选择支持的最近ubuntu版本17.10,下载适用于zabbix的安装包。
3.5.2.2.2:安装
安装:
root@zabbix-mysql:~# dpkg -i percona-zabbix-templates_1.1.8-1.artful_all.deb
将安装生成的UserParameter配置文件拷贝到zabbix-agent的配置目录下,其中定义了监控模板中使用到的监控键值。
root@zabbix-mysql:~# cp /var/lib/zabbix/percona/templates/userparameter_percona_mysql.conf /etc/zabbix/zabbix_agentd.d/
Ensure /etc/zabbix_agentd.conf contains the line:
Include=/etc/zabbix_agentd.conf.d/
确保zabbix-agent的主配置文件中设置了Include=/etc/zabbix_agentd.conf.d/,保证模块化的配置文件能够生效。
重启zabbix-agent使配置生效:
root@zabbix-mysql:~# systemctl restart zabbix-agent
3.5.2.2.3:MySQL-Master 安装 php5.6
由于本次实验的系统版本是Ubuntu18.04,和插件支持的版本17.10不符,需要将php版本降级到与其匹配的5.6。
root@zabbix-mysql:~# add-apt-repository ppa:ondrej/php
root@zabbix-mysql:~# apt update
root@zabbix-mysql:~# apt install php5.6 php5.6-mysql
如果不安装php5.6,执行监控脚本时会报如下错误:
root@zabbix-mysql:~# /var/lib/zabbix/percona/scripts/get_mysql_stats_wrapper.sh gg /var/lib/zabbix/percona/scripts/get_mysql_stats_wrapper.sh: 35: /var/lib/zabbix/percona/scripts/get_mysql_stats_wrapper.sh: /usr/bin/php: not found ERROR: run the command manually to investigate the problem: /usr/bin/php -q /var/lib/zabbix/percona/scripts/ss_get_mysql_stats.php --host localhost --items gg
3.5.2.2.4:创建连接 MySQL 的配置文件
Create .cnf file /var/lib/zabbix/percona/scripts/ss_get_mysql_stats.php.cnf as described at configuration file:
在其中配置脚本监测时所用的mysql用户和密码:
root@zabbix-mysql:~# vim /var/lib/zabbix/percona/scripts/ss_get_mysql_stats.php.cnf
<?php
$mysql_user = 'root';
$mysql_pass = '123456';
3.5.2.2.5:测试监测脚本
能获取到数值,表明脚本可以正常执行并获取数据:
root@zabbix-mysql:~# /var/lib/zabbix/percona/scripts/get_mysql_stats_wrapper.sh gg
40
3.5.2.3:Zabbix Server 导入模板并测试
3.5.2.3.1:导入模板
插件安装后,Percona提供的监控模板xml文件会在如下路径:
/var/lib/zabbix/percona/templates/zabbix_agent_template_percona_mysql_server_ht_2.0.9-sver1.1.8.xml
但经过测试,这个模板无法导入zabbix,这里使用的是马哥教育的张士杰(杰哥)老师修改后的模板,可以正常导入:
3.5.2.3.2:关联主机并验证
将模板关联至node100,即MySQL-Master:
验证最新数据:
3.5.3:MySQL 主从同步状态监测脚本
基于slave节点的SHOW SLAVE STATUS\G
信息来获取主从同步的状态。
3.5.3.1:SHOW SLAVE STATUS
主从同步状态需要关注以下指标:
-
Slave_IO_Running:IO_TREAD的运行状态,yes表示与Master的二进制日志复制正常,no为异常。
-
Slave_SQL_Running:SQL_THREAD的运行状态,yes表示中继日志写入数据库正常,no为异常。
-
Last_Errno:Slave的复制线程的最近一次错误代码,0表示最近Last_IO_Errno和Last_IO_Errno都没有错误代码产生。
Last_IO_Errno和Last_SQL_Errno任意一个出现错误代码,Last_Errno会显示二者中较近出现的错误代码。
监控时只需要监控后面的IO和SQL错误代码状态即可。 -
Last_IO_Errno:IO_TREAD的最近一次错误代码,0为正常,非0表示发生了错误。
-
Last_SQL_Errno:SQL_TREAD的最近一次错误代码,0为正常,非0表示发生了错误。
-
Seconds_Behind_Master:反映主从同步的延迟,是Master和Slave对同一事件的时间戳记录的差值。
由于Master的worker线程在写binlog时是并发的,而主从复制时,Master的dump线程、Slave的IO和SQL线程都是单一线程,加上正常同步复制需要考虑的网络延迟、服务器性能差异等,在高并发的业务量下,Slave往往会与Master产生较大延迟,对数据完整性和安全性造成隐患,所以这是一个重要的监控指标。
SHOW SLAVE STATUS 完整信息:
mysql> SHOW SLAVE STATUS\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.1.100
Master_User: repl
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000005
Read_Master_Log_Pos: 2716883
Relay_Log_File: mysql-relay.000005
Relay_Log_Pos: 2717096
Relay_Master_Log_File: mysql-bin.000005
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 2716883
Relay_Log_Space: 2717776
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 100
Master_UUID: b5940e14-7bed-11eb-8e0f-000c298c58ca
Master_Info_File: /var/lib/mysql/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set:
Auto_Position: 0
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.01 sec)
3.5.3.2:创建用于检查 MySQL 的用户
由于zabbix server是通过远程调用脚本的,而MySQL的root用户在初始安全设置时禁用了远程登录,所以单独创建一个用户,用于脚本监测:
mysql> GRANT ALL ON *.* TO 'check'@'192.168.1.%' IDENTIFIED BY '123456';
mysql> FLUSH PRIVILEGES;
3.5.3.3:编写脚本
#!/bin/bash
# Description: MySQL主从同步状态监测脚本
# Author: yqc
# Variables set
Mysql_User=check
Mysql_Passwd=123456
Mysql_Addr=192.168.1.200
Mysql_Port=3306
Item_Name=$1
# Define function "Slave_Running"
Slave_Running(){
IO_Running=$(mysql -u${Mysql_User} -p${Mysql_Passwd} -h${Mysql_Addr} -P${Mysql_Port} -e "SHOW SLAVE STATUS\G" 2> /dev/null | awk -F': ' '/Slave_IO_Running:/{print $2}')
SQL_Running=$(mysql -u${Mysql_User} -p${Mysql_Passwd} -h${Mysql_Addr} -P${Mysql_Port} -e "SHOW SLAVE STATUS\G" 2> /dev/null | awk -F': ' '/Slave_SQL_Running:/{print $2}')
if [[ ${IO_Running} == "Yes" && ${SQL_Running} == "Yes" ]]; then
Slave_Running=1
else
Slave_Running=0
fi
echo ${Slave_Running}
}
# Define function "Last_IO_Status"
Last_IO_Status(){
Last_IO_Errno=$(mysql -u${Mysql_User} -p${Mysql_Passwd} -h${Mysql_Addr} -P${Mysql_Port} -e "SHOW SLAVE STATUS\G" 2> /dev/null | awk -F': ' '/Last_IO_Errno:/{print $2}')
if [[ ${Last_IO_Errno} = 0 ]]; then
Last_IO_Status=1
else
Last_IO_Status=0
fi
echo ${Last_IO_Status}
}
# Define function "Last_SQL_Status"
Last_SQL_Status(){
Last_SQL_Errno=$(mysql -u${Mysql_User} -p${Mysql_Passwd} -h${Mysql_Addr} -P${Mysql_Port} -e "SHOW SLAVE STATUS\G" 2> /dev/null | awk -F': ' '/Last_SQL_Errno:/{print $2}')
if [[ ${Last_SQL_Errno} = 0 ]]; then
Last_SQL_Status=1
else
Last_SQL_Status=0
fi
echo ${Last_SQL_Status}
}
# Define function “Seconds_Behind_Master"
Seconds_Behind_Master(){
mysql -u${Mysql_User} -p${Mysql_Passwd} -h${Mysql_Addr} -P${Mysql_Port} -e "SHOW SLAVE STATUS\G" 2> /dev/null | awk -F': ' '/Seconds_Behind_Master:/{print $2}'
}
# Define function "Main"
Main(){
case ${Item_Name} in
sr)
Slave_Running;;
is)
Last_IO_Status;;
ss)
Last_SQL_Status;;
sb)
Seconds_Behind_Master;;
esac
}
# Use function "Main"
Main $1
3.5.3.4:测试脚本
# 为1表示IO和SQL线程运行状态均为Yes,为0表示异常
root@zabbix-mysql-backup:~# bash Mysql_Slave_Status.sh sr
1
# 为1,表示最近的IO正常,0表示有错误代码
root@zabbix-mysql-backup:~# bash Mysql_Slave_Status.sh is
1
# 为1,表示最近的SQL正常,0表示有错误代码
root@zabbix-mysql-backup:~# bash Mysql_Slave_Status.sh ss
1
# Seconds_Behind_Master的秒数
root@zabbix-mysql-backup:~# bash Mysql_Slave_Status.sh sb
0
3.5.4:Zabbix Agent 添加 UserParameter
3.5.4.1:MySQL-Slave 安装 zabbix-agent
3.5.4.1.1:安装
root@zabbix-mysql-backup:~# wget https://repo.zabbix.com/zabbix/4.0/ubuntu/pool/main/z/zabbix-release/zabbix-release_4.0-3+bionic_all.deb
root@zabbix-mysql-backup:~# dpkg -i zabbix-release_4.0-3+bionic_all.deb
root@zabbix-mysql-backup:~# sed -i 's@repo.zabbix.com/zabbix/4.0@mirrors.aliyun.com/zabbix/zabbix/4.0@g' /etc/apt/sources.list.d/zabbix.list
root@zabbix-mysql-backup:~# apt update
root@zabbix-mysql-backup:~# apt install zabbix-agent
3.5.4.1.2:配置
root@zabbix-mysql-backup:~# grep '^[a-Z]' /etc/zabbix/zabbix_agentd.conf
PidFile=/var/run/zabbix/zabbix_agentd.pid
LogFile=/var/log/zabbix/zabbix_agentd.log
LogFileSize=0
EnableRemoteCommands=1
LogRemoteCommands=1
Server=192.168.1.101
ListenPort=10050
StartAgents=5
ServerActive=192.168.1.101
Hostname=192.168.1.200
User=zabbix
Include=/etc/zabbix/zabbix_agentd.d/*.conf
3.5.4.1.3:重启
root@zabbix-mysql-backup:~# systemctl restart zabbix-agent
3.5.4.2:添加 UserParameter
拷贝脚本并添加执行权限:
root@zabbix-mysql-backup:~# cp /root/Mysql_Slave_Status.sh /etc/zabbix/zabbix_agentd.d/
root@zabbix-mysql-backup:~# chmod +x /etc/zabbix/zabbix_agentd.d/Mysql_Slave_Status.sh
添加UserParameter:
root@zabbix-mysql-backup:~# vim /etc/zabbix/zabbix_agentd.d/Mysql_Slave_Status.conf
UserParameter=mysql_slave_status[*],/etc/zabbix/zabbix_agentd.d/Mysql_Slave_Status.sh "$1"
重启:
root@zabbix-mysql-backup:~# systemctl restart zabbix-agent
3.5.4.3:Zabbix Server 测试手动获取监控数据
zabbix_get手动获取数据。
root@zabbix-server:~# zabbix_get -s 192.168.1.200 -p 10050 -k "mysql_slave_status[ss]"
1
root@zabbix-server:~# zabbix_get -s 192.168.1.200 -p 10050 -k "mysql_slave_status[is]"
1
root@zabbix-server:~# zabbix_get -s 192.168.1.200 -p 10050 -k "mysql_slave_status[ss]"
1
root@zabbix-server:~# zabbix_get -s 192.168.1.200 -p 10050 -k "mysql_slave_status[sr]"
1
root@zabbix-server:~# zabbix_get -s 192.168.1.200 -p 10050 -k "mysql_slave_status[sb]"
0
3.5.5:Zabbix Server 制作 MySQL 主从同步状态监测模板
3.5.5.1:创建模板
3.5.5.2:创建应用集
3.5.5.3:创建监控项
创建Slave线程运行状况的监控项:
依次克隆修改,创建其它监控项:
3.5.6:Zabbix Server 添加主机并关联模板
3.5.6.1:添加 MySQL-Slave 为被监控主机
3.5.6.2:关联模板
3.5.6.3:验证监控数据
最新数据: