canal mysql mongodb_使用Percona2监控MongoDB,MySQL-CSDN博客

本文链接：https://blog.csdn.net/weixin_36367702/article/details/113892736

本文档详细介绍了如何使用Percona Monitoring and Management (PMM) 2来监控MySQL和MongoDB集群。首先，通过Docker部署PMM Server，并确保其正确运行。然后，在客户端服务器上安装PMM Client，初始化配置并注册MongoDB和MySQL实例。最后，通过调整MongoDB的Profiler设置和Grafana仪表板以实现有效的监控。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

1. server端

下载镜像:

[root@hlet-prod-k8s-rancher ~]# docker pull percona/pmm-server:2 # 下载镜像

Pulling from percona/pmm-server

524b0c1e57f8: Pull complete

e69c94edc13d: Pull complete

Digest: sha256:799651e0715f106a92632072eb34a350a2f25630622dda0680987fa13b4fd396

Status: Downloaded newer image for percona/pmm-server:2

docker.io/percona/pmm-server:2

查看端口,由于80,443端口已经被占用,这里使用8080,8443端口

[root@hlet-prod-k8s-rancher ~]# netstat -nptl

Active Internet connections (only servers)

Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name

tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1765/sshd

tcp 0 0 127.0.0.1:40450 0.0.0.0:* LISTEN 20377/node

tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/systemd

tcp6 0 0 :::5557 :::* LISTEN 25413/docker-proxy

tcp6 0 0 :::22 :::* LISTEN 1765/sshd

tcp6 0 0 :::443 :::* LISTEN 10107/docker-proxy

tcp6 0 0 :::8001 :::* LISTEN 25577/docker-proxy

tcp6 0 0 :::111 :::* LISTEN 1499/rpcbind

tcp6 0 0 :::80 :::* LISTEN 10122/docker-proxy

创建数据卷

[root@hlet-prod-k8s-rancher ~]# docker create \

\> -v /srv \

\> --name pmm-data \

\> percona/pmm-server:2 /bin/true

f6f0d8e27098e376f9156711c5eb3db77049a4c18f7a30a546be6ab4ec381305 # 创建数据卷

运行server的镜像并挂载数据卷

[root@hlet-prod-k8s-rancher ~]# docker run -d \

\> -p 8080:80 \

\> -p 8443:443 \

\> --volumes-from pmm-data \

\> --name pmm-server \

\> --restart always \

\> percona/pmm-server:2

7b2ed49138407e75962e0a56a573e83882cc4f72636a107d1f28e70b0895e1d9 # 运行pmm server并挂载数据卷

查看是否部署成功:

[root@hlet-prod-k8s-rancher ~]# docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

7b2ed4913840 percona/pmm-server:2 "/opt/entrypoint.sh" 16 seconds ago Up 16 seconds (healthy) 0.0.0.0:8080->80/tcp, 0.0.0.0:8443->443/tcp pmm-server

047eac0cc3e4 rancher/rancher:v2.4.5 "entrypoint.sh" 2 weeks ago Up 2 weeks 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp unruffled_tesla

b956d530ede2 styletang/rocketmq-console-ng "sh -c 'java $JAVA_O…" 2 weeks ago Up 2 weeks 0.0.0.0:8001->8080/tcp reverent_ellis

434ae88e371d 10.1.99.42/monitor/rocketmq-exporter:latest "java -jar quickstar…" 2 weeks ago Up 2 weeks 0.0.0.0:5557->5557/tcp relaxed_kalam

尝试访问:

[root@hlet-prod-k8s-rancher ~]# curl --insecure https://10.1.99.50:8443/swagger/ # 测试服务是否启动完成

初始化server:

访问http://10.1.99.50:8080/graph/ 或者https://10.1.99.50:8443/graph/,默认初始用户名和密码均为admin,登陆后会要求重置密码

2.client端

2.1 安装

这里我们MongoDB和mysql的服务器都是基于centos的,yum或rpm方式选一种即可

yum方式安装:

yum install -y https://repo.percona.com/yum/percona-release-latest.noarch.rpm

yum install pmm2-client -y

如果下载太慢的话,也可以直接下载rpm包来安装,地址:

https://repo.percona.com/yum/release/2/RPMS/x86_64/pmm2-client-2.8.0-6.el7.x86_64.rpm

rpm方式安装:

rpm -ivh /home/pmm2-client-2.8.0-6.el7.x86_64.rpm

2.2 初始化

向PMM server注册当前节点:

这里url需要根据情况填写自己初始化的PMM server的密码

pmm-admin config --server-insecure-tls --server-url=https://admin:'yourpassword'@10.1.99.50:8443

如果想在注册的时候需要指定node name,可以参考

Positional arguments:

[] Node address (autodetected default: 10.1.30.35)

[] Node type, one of: generic, container (default: generic)

[] Node name (autodetected default: localhost.localdomain)

实际写法: 这里第三个参数node name我直接指定成了IP

pmm-admin config 10.1.30.35 generic 10.1.30.35 --server-insecure-tls --server-url=https://admin:'yourpassword'@10.1.99.50:8443

如果服务器启用了firewalld,需要对server开放相应的端口

1.编辑firewalld的service

vi /etc/firewalld/services/pmm-agent.xml

# 写入以下内容,其中端口范围可以在PMM的配置文件中查到/usr/local/percona/pmm2/config/pmm-agent.yaml

pmm-agent

PMM agent

2.添加策略

其中source address是PMM server的IP地址段

firewall-cmd --permanent --zone=public --add-rich-rule="rule family="ipv4" source address="10.1.99.0/24" service name="pmm-agent" accept"

firewall-cmd --reload

查看策略:

[root@localhost ~]# firewall-cmd --zone=public --list-all

public (active)

target: default

icmp-block-inversion: no

interfaces: eno1

sources:

services: ssh dhcpv6-client ntp

ports: 20010-20011/tcp 20003/tcp 10050/tcp 9216/tcp 9100/tcp

protocols:

masquerade: no

forward-ports:

source-ports:

icmp-blocks:

rich rules:

rule family="ipv4" source address="10.1.99.0/24" service name="pmm-agent" accept # 可以看到刚添加的策略

2.3 注册MongoDB

Mongodb 部署信息,mongos部署两个点,config3个点,和3个分片集群

Primiary

Secondary

Arbiter

mongos

10.1.99.77 10.1.99.78

config

10.1.99.72

10.1.99.74 10.1.99.76

shard1

10.1.99.71

10.1.99.72

10.1.99.77

shard2

10.1.99.73

10.1.99.74

10.1.99.78

shard3

10.1.99.75

10.1.99.76

10.1.99.77

注意这里的用户名密码建议额外创建一下

注意这边如果有多个mongos服务,需要选择其中的一个打上标签 type=mongos ,不然监控详情页有的图标会因为重复收集而报错

10.1.99.71

pmm-admin add mongodb --username=mongoroot --password=mongopassword --service-name=mongo-4.0-rs1-primary --host=10.1.99.71 --port=20001 --cluster=mongo-4.0 --replication-set=shard1 --environment=4.0

10.1.99.72

pmm-admin add mongodb --username=mongoroot --password=mongopassword --service-name=mongo-4.0-rs1-seconday --host=10.1.99.72 --port=20001 --cluster=mongo-4.0 --replication-set=shard1 --environment=4.0

pmm-admin add mongodb --username=mongoroot --password=mongopassword --service-name=mongo-4.0-config-01 --host=10.1.99.72 --port=20000 --cluster=mongo-4.0 --replication-set=configs --environment=4.0

10.1.99.73

pmm-admin add mongodb --username=mongoroot --password=mongopassword --service-name=mongo-4.0-rs2-primary --host=10.1.99.73 --port=20002 --cluster=mongo-4.0 --replication-set=shard2 --environment=4.0

10.1.99.74

pmm-admin add mongodb --username=mongoroot --password=mongopassword --service-name=mongo-4.0-rs2-seconday --host=10.1.99.74 --port=20002 --cluster=mongo-4.0 --replication-set=shard2 --environment=4.0

pmm-admin add mongodb --username=mongoroot --password=mongopassword --service-name=mongo-4.0-config-02 --host=10.1.99.74 --port=20000 --cluster=mongo-4.0 --replication-set=configs --environment=4.0

10.1.99.75

pmm-admin add mongodb --username=mongoroot --password=mongopassword --service-name=mongo-4.0-rs3-primary --host=10.1.99.75 --port=20003 --cluster=mongo-4.0 --replication-set=shard3 --environment=4.0

10.1.99.76

pmm-admin add mongodb --username=mongoroot --password=mongopassword --service-name=mongo-4.0-rs3-seconday --host=10.1.99.76 --port=20003 --cluster=mongo-4.0 --replication-set=shard3 --environment=4.0

pmm-admin add mongodb --username=mongoroot --password=mongopassword --service-name=mongo-4.0-config-03 --host=10.1.99.76 --port=20000 --cluster=mongo-4.0 --replication-set=configs --environment=4.0

10.1.99.77

pmm-admin add mongodb --username=mongoroot --password=mongopassword --service-name=mongo-4.0-mongos-01 --host=10.1.99.77 --port=27017 --cluster=mongo-4.0 --environment=4.0

pmm-admin add mongodb --service-name=mongo-4.0-rs1-arbiter --host=127.0.0.1 --port=20001 --cluster=mongo-4.0 --replication-set=shard1 --environment=4.0

pmm-admin add mongodb --service-name=mongo-4.0-rs3-arbiter --host=127.0.0.1 --port=20003 --cluster=mongo-4.0 --replication-set=shard3 --environment=4.0

10.1.99.78 # 打上标签 type=mongos

pmm-admin add mongodb --username=mongoroot --password=mongopassword --service-name=mongo-4.0-mongos-02 --host=10.1.99.78 --port=27017 --cluster=mongo-4.0 --environment=4.0 --custom-labels="type=mongos"

pmm-admin add mongodb --service-name=mongo-4.0-rs2-arbiter --host=127.0.0.1 --port=20002 --cluster=mongo-4.0 --replication-set=shard2 --environment=4.0

查看已经注册的服务:

[root@hlet-prod-mongo-07 log]# pmm-admin list

Service type Service name Address and port Service ID

MongoDB hlet-prod-mongo-07 10.1.99.77:27017 /service_id/333dc904-914b-4c15-a233-5da8cf16b11a

Agent type Status Agent ID Service ID

pmm_agent Connected /agent_id/aa6b8277-f91d-46b8-95dd-d131427a16e8

node_exporter Running /agent_id/7030efbb-ddb4-4334-8441-31991a8fc48b

mongodb_exporter Running /agent_id/6753b361-f46b-4c5d-b1d4-72b3c1346835 /service_id/333dc904-914b-4c15-a233-5da8cf16b11a

mongodb_profiler_agent Running /agent_id/3e8b27c9-a54f-459e-ab62-e6b8f2775da3 /service_id/333dc904-914b-4c15-a233-5da8cf16b11a

如果需要删除已经注册的服务:

pmm-admin remove mongodb hlet-prod-mongo-07 # hlet-prod-mongo-07 是对应上面的service name

至此MongoDB服务注册完成,在PMM的grafana页面就可以看到数据了

开启性能分析

在所有非仲裁节点执行开启Profiling设置

shard2:SECONDARY> db.getProfilingLevel()

shard2:SECONDARY> db.setProfilingLevel( 1 , 200 );

{

"was" : 0,

"slowms" : 100,

"sampleRate" : 1,

"ok" : 1,

"$gleStats" : {

"lastOpTime" : Timestamp(0, 0),

"electionId" : ObjectId("000000000000000000000000")

"lastCommittedOpTime" : Timestamp(1594806686, 1),

"$configServerState" : {

"opTime" : {

"ts" : Timestamp(1594806672, 2),

"t" : NumberLong(7)

}

"$clusterTime" : {

"clusterTime" : Timestamp(1594806687, 1),

"signature" : {

"hash" : BinData(0,"4Mn8aBf+qYiPfodrhug1oWTVXdQ="),

"keyId" : NumberLong("6831814880710885409")

}

"operationTime" : Timestamp(1594806686, 1)

}

查看开启状态:

shard3:SECONDARY> db.getProfilingStatus()

{

"was" : 1,

"slowms" : 200,

"sampleRate" : 1,

"$gleStats" : {

"lastOpTime" : Timestamp(0, 0),

"electionId" : ObjectId("000000000000000000000000")

"lastCommittedOpTime" : Timestamp(1594806737, 1),

"$configServerState" : {

"opTime" : {

"ts" : Timestamp(1594806735, 2),

"t" : NumberLong(7)

}

"$clusterTime" : {

"clusterTime" : Timestamp(1594806741, 1),

"signature" : {

"hash" : BinData(0,"I7pYRfCoF8eqrbiYa4yFnj+JyKo="),

"keyId" : NumberLong("6831814880710885409")

}

"operationTime" : Timestamp(1594806737, 1)

}

调整MongoDB的dashboard:

在前面注册服务的时候,有提示对集群中的一个mongos打标签 type=mongos,这是因为在MongoDB的集群总览表中,由于默认的查询语句会导致下面两个表会获取mongos数量份数的数据,导致展示异常,整个表变成红色,所有值显式为重复的值.

ec65c637e1db

解决思路是grafana的图标在做查询时,添加一个基于标签的条件,即type=mongos,这样就不会重复获取了.

操作步骤:

1.在MongoDB cluster summary 这张表中点击齿轮图标

ec65c637e1db

2.开启编辑模式:

ec65c637e1db

3.再从顶部的导航点到MongoDB cluster summary页面

ec65c637e1db

这个时候点击图表名称后面的小三角,可以看到图标成了可编辑的状态

ec65c637e1db

点击edit进去后,编辑查询语句,添加查询条件type="mongos" :

mongodb_mongos_db_collections_total{cluster="$cluster",db!~"admin|config",type="mongos"}

ec65c637e1db

修改完成后,图表应该就变成蓝色,并能正常显示数据了.

同样的操作对Size of Collections in Shards表也执行一下:

ec65c637e1db

查询语句:

mongodb_mongos_db_data_size_bytes{cluster="$cluster",db!~"admin|config",type="mongos"}

ec65c637e1db

两个表格都改完后,点击右上角的保存按钮保存

ec65c637e1db

保存时选择覆盖原表:

ec65c637e1db

2.4 注册mysql

2.4.1 注册proxysql

如果没有部署proxysql可以跳过

rpm -ivh /home/pmm2-client-2.8.0-6.el7.x86_64.rpm

pmm-admin config --server-insecure-tls --server-url=https://admin:'yourpassword'@10.1.99.50:8443

pmm-admin add proxysql --username=admin --password=admin --environment=4.0 --cluster=mysql-4.0 # proxysql管理端口的用户名密码

2.4.2 注册mysql node

mysql-01:

[root@hlet-mysql-01 ~]# rpm -ivh /home/pmm2-client-2.8.0-6.el7.x86_64.rpm

警告：/home/pmm2-client-2.8.0-6.el7.x86_64.rpm: 头V4 RSA/SHA256 Signature, 密钥 ID 8507efa5: NOKEY

准备中... ################################# [100%]

正在升级/安装...

1:pmm2-client-2.8.0-6.el7 ################################# [100%]

[root@hlet-mysql-01 ~]# pmm-admin config --server-insecure-tls --server-url=https://admin:'yourpassword'@10.1.99.50:8443

Checking local pmm-agent status...

pmm-agent is running.

Registering pmm-agent on PMM Server...

Registered.

Configuration file /usr/local/percona/pmm2/config/pmm-agent.yaml updated.

Reloading pmm-agent configuration...

Configuration reloaded.

Checking local pmm-agent status...

pmm-agent is running.

[root@hlet-mysql-01 ~]# pmm-admin add mysql --query-source=slowlog --username=yourmysqluser --password=yourmysqlpassword --environment=4.0 --cluster=mysql-4.0 10.1.99.26 127.0.0.1:3306

MySQL Service added.

Service ID : /service_id/fb51fb7c-a3f9-4a02-9cad-39f94ac949ac

Service name: 10.1.99.26

Table statistics collection enabled (the limit is 1000, the actual table count is 636).

mysql-02:

[root@hlet-mysql-02 ~]# rpm -ivh /home/pmm2-client-2.8.0-6.el7.x86_64.rpm

警告：/home/pmm2-client-2.8.0-6.el7.x86_64.rpm: 头V4 RSA/SHA256 Signature, 密钥 ID 8507efa5: NOKEY

准备中... ################################# [100%]

正在升级/安装...

1:pmm2-client-2.8.0-6.el7 ################################# [100%]

[root@hlet-mysql-02 ~]# pmm-admin config --server-insecure-tls --server-url=https://admin:'yourpassword'@10.1.99.50:8443

Checking local pmm-agent status...

pmm-agent is running.

Registering pmm-agent on PMM Server...

Registered.

Configuration file /usr/local/percona/pmm2/config/pmm-agent.yaml updated.

Reloading pmm-agent configuration...

Configuration reloaded.

Checking local pmm-agent status...

pmm-agent is running.

[root@hlet-mysql-02 ~]# pmm-admin add mysql --query-source=slowlog --username=yourmysqluser --password=yourmysqlpassword --environment=4.0 --cluster=mysql-4.0 10.1.99.27 127.0.0.1:3306

MySQL Service added.

Service ID : /service_id/e588e99e-0937-4027-ad0a-1fe647f5320c

Service name: 10.1.99.27

Table statistics collection enabled (the limit is 1000, the actual table count is 636).

hlet-mysql-03:

[root@hlet-mysql-03 ~]# rpm -ivh /home/pmm2-client-2.8.0-6.el7.x86_64.rpm

警告：/home/pmm2-client-2.8.0-6.el7.x86_64.rpm: 头V4 RSA/SHA256 Signature, 密钥 ID 8507efa5: NOKEY

准备中... ################################# [100%]

正在升级/安装...

1:pmm2-client-2.8.0-6.el7 ################################# [100%]

[root@hlet-mysql-03 ~]# pmm-admin config --server-insecure-tls --server-url=https://admin:'yourpassword'@10.1.99.50:8443

Checking local pmm-agent status...

pmm-agent is running.

Registering pmm-agent on PMM Server...

Registered.

Configuration file /usr/local/percona/pmm2/config/pmm-agent.yaml updated.

Reloading pmm-agent configuration...

Configuration reloaded.

Checking local pmm-agent status...

pmm-agent is running.

[root@hlet-mysql-03 ~]# pmm-admin add mysql --query-source=slowlog --username=yourmysqluser --password=yourmysqlpassword --environment=4.0 --cluster=mysql-4.0 10.1.99.28 127.0.0.1:3306

MySQL Service added.

Service ID : /service_id/a0f795f0-480f-45fc-bfd8-1a54182a4bc9

Service name: 10.1.99.28

Table statistics collection enabled (the limit is 1000, the actual table count is 636).

3. percona-toolkit工具使用

3.1 安装

可用于mysql的慢查询分析

下载地址:

登陆mysql后,安装:

yum localinstall percona-toolkit-3.2.0-1.el7.x86_64.rpm -y

3.2 慢查询分析

使用的是pt-query-digest 命令