六、Prometheus+Grafana搭建监控系统

达摩院高僧

已于 2022-07-02 23:31:46 修改

阅读量587

点赞数

分类专栏：分布式容器化架构搭建专题文章标签：数据库 java mysql

于 2022-07-02 23:21:51 首次发布

本文链接：https://blog.csdn.net/ahwuzebao/article/details/125001507

版权

分布式容器化架构搭建专题专栏收录该内容

12 篇文章 0 订阅

订阅专栏

监控系统简介

Prometheus是一个开源系统监控和警报工具包，最初在 SoundCloud构建。自 2012 年成立以来，许多公司和组织都采用了 Prometheus，该项目拥有非常活跃的开发者和用户社区。它现在是一个独立的开源项目，独立于任何公司维护。为了强调这一点，并明确项目的治理结构，Prometheus 于 2016 年加入云原生计算基金会，成为继Kubernetes之后的第二个托管项目。

Prometheus 将其指标收集并存储为时间序列数据，即指标信息与记录时的时间戳以及称为标签的可选键值对一起存储。

特征
普罗米修斯的主要特点是：

具有由度量名称和键/值对标识的时间序列数据的多维数据模型
PromQL，一种利用这种维度的灵活查询语言
不依赖分布式存储；单个服务器节点是自治的
时间序列收集通过 HTTP 上的pull 模型进行
通过中间网关支持推送时间序列
通过服务发现或静态配置发现目标
多种图形模式和仪表板支持

Prometheus通过安装在远程机器上的exporter来收集监控数据，这里用到了以下两个exporter：

node_exporter – 用于机器系统数据
mysqld_exporter – 用于Mysql服务器数据

Grafana是一个开源的功能丰富的数据可视化平台，通常用于时序数据的可视化

下载

wget https://github.com/prometheus/prometheus/releases/download/v2.36.0-rc0/prometheus-2.36.0-rc0.linux-amd64.tar.gz

wget https://github.com/prometheus/mysqld_exporter/releases/download/v0.14.0/mysqld_exporter-0.14.0.linux-amd64.tar.gz

wget https://github.com/prometheus/node_exporter/releases/download/v1.3.1/node_exporter-1.3.1.linux-amd64.tar.gz

wget https://dl.grafana.com/enterprise/release/grafana-enterprise-9.0.2.linux-amd64.tar.gz

安装启动prometheus

先解压启动node_exporter 和 mysqld_exporter

[root@host125 install]# tar -xvf node_exporter-1.3.1.linux-amd64.tar.gz 
[root@host125 install]# tar -xvf mysqld_exporter-0.14.0.linux-amd64.tar.gz
[root@host125 install]# cd node_exporter-1.3.1.linux-amd64/
[root@host125 node_exporter-1.3.1.linux-amd64]# nohup ./node_exporter &

##启动mysqld_exporter之前需要先创建一个mysql用户并赋权
[root@host125 log]# mysql -uroot -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 8
Server version: 8.0.27 MySQL Community Server - GPL

Copyright (c) 2000, 2021, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> CREATE USER 'exporter'@'localhost' IDENTIFIED BY '123456' WITH MAX_USER_CONNECTIONS 3;
Query OK, 0 rows affected (0.03 sec)

mysql> GRANT PROCESS, REPLICATION CLIENT, SELECT ON *.* TO 'exporter'@'localhost';
Query OK, 0 rows affected (0.00 sec)

[root@host125 install]# cd mysqld_exporter-0.14.0.linux-amd64/
##启动mysqld_exporter之前需要先创建.my.cnf配置文件，配置内容为mysql用户名密码
[root@host125 mysqld_exporter-0.14.0.linux-amd64]# vi .my.cnf
[client]
user=exporter
password=123456
[root@host125 mysqld_exporter-0.14.0.linux-amd64]# nohup ./mysqld_exporter --config.my-cnf=.my.cnf &
[root@host125 mysqld_exporter-0.14.0.linux-amd64]# vi nohup.out
ts=2022-07-02T15:03:23.135Z caller=mysqld_exporter.go:277 level=info msg="Starting mysqld_exporter" version="(version=0.14.0, branch=HEAD, revision=ca1b9af82a471c849c529eb8aadb1aac73e7b68c)"
ts=2022-07-02T15:03:23.135Z caller=mysqld_exporter.go:278 level=info msg="Build context" (gogo1.17.8,userroot@401d370ca42e,date20220304-16:25:15)=(MISSING)
ts=2022-07-02T15:03:23.135Z caller=mysqld_exporter.go:293 level=info msg="Scraper enabled" scraper=global_variables
ts=2022-07-02T15:03:23.135Z caller=mysqld_exporter.go:293 level=info msg="Scraper enabled" scraper=slave_status
ts=2022-07-02T15:03:23.135Z caller=mysqld_exporter.go:293 level=info msg="Scraper enabled" scraper=global_status
ts=2022-07-02T15:03:23.135Z caller=mysqld_exporter.go:293 level=info msg="Scraper enabled" scraper=info_schema.innodb_cmp
ts=2022-07-02T15:03:23.136Z caller=mysqld_exporter.go:293 level=info msg="Scraper enabled" scraper=info_schema.innodb_cmpmem
ts=2022-07-02T15:03:23.136Z caller=mysqld_exporter.go:293 level=info msg="Scraper enabled" scraper=info_schema.query_response_time
ts=2022-07-02T15:03:23.136Z caller=mysqld_exporter.go:303 level=info msg="Listening on address" address=:9104
ts=2022-07-02T15:03:23.137Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false

再解压启动prometheus，注意启动前先修改prometheus.yml配置，修改配置时要注意格式（缩进2个空格）

[root@host125 install]# tar -xvf prometheus-2.36.0-rc0.linux-amd64.tar.gz
[root@host125 install]# chown -R root:root prometheus-2.36.0-rc0.linux-amd64
[root@host125 install]# cd prometheus-2.36.0-rc0.linux-amd64/
[root@host125 prometheus-2.36.0-rc0.linux-amd64]# vi prometheus.yml
# my global config
global:
  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
    - static_configs:
        - targets:
          # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: "prometheus"

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ["localhost:9090"]
  - job_name: 'node'
    static_configs:
      - targets: ['localhost:9100']
  - job_name: 'mysql'
    static_configs:
      - targets: ['localhost:9104']
[root@host125 prometheus-2.36.0-rc0.linux-amd64]# nohup ./prometheus &
[root@host125 prometheus-2.36.0-rc0.linux-amd64]# vi nohup.out
ts=2022-07-02T10:06:45.124Z caller=main.go:491 level=info msg="No time or size retention was set so using the default time retention" duration=15d
ts=2022-07-02T10:06:45.124Z caller=main.go:535 level=info msg="Starting Prometheus Server" mode=server version="(version=2.36.0-rc0, branch=HEAD, revision=ee37387a7d48a286d3a79c510cd3764d87ccf18a)"
ts=2022-07-02T10:06:45.124Z caller=main.go:540 level=info build_context="(go=go1.18.2, user=root@97451218497e, date=20220525-12:27:02)"
ts=2022-07-02T10:06:45.124Z caller=main.go:541 level=info host_details="(Linux 4.18.0-240.10.1.el8_3.x86_64 #1 SMP Mon Jan 18 17:05:51 UTC 2021 x86_64 host125 (none))"
ts=2022-07-02T10:06:45.124Z caller=main.go:542 level=info fd_limits="(soft=1024, hard=262144)"
ts=2022-07-02T10:06:45.124Z caller=main.go:543 level=info vm_limits="(soft=unlimited, hard=unlimited)"
ts=2022-07-02T10:06:45.155Z caller=web.go:553 level=info component=web msg="Start listening for connections" address=0.0.0.0:9090
ts=2022-07-02T10:06:45.157Z caller=main.go:972 level=info msg="Starting TSDB ..."
ts=2022-07-02T10:06:45.159Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false
ts=2022-07-02T10:06:45.166Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any"
ts=2022-07-02T10:06:45.166Z caller=head.go:536 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.661µs
ts=2022-07-02T10:06:45.166Z caller=head.go:542 level=info component=tsdb msg="Replaying WAL, this may take a while"
ts=2022-07-02T10:06:45.166Z caller=head.go:613 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0
ts=2022-07-02T10:06:45.166Z caller=head.go:619 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=60.297µs wal_replay_duration=250.735µs total_replay_duration=358.859µs
ts=2022-07-02T10:06:45.167Z caller=main.go:993 level=info fs_type=XFS_SUPER_MAGIC
ts=2022-07-02T10:06:45.167Z caller=main.go:996 level=info msg="TSDB started"
ts=2022-07-02T10:06:45.167Z caller=main.go:1177 level=info msg="Loading configuration file" filename=prometheus.yml
ts=2022-07-02T10:06:45.326Z caller=main.go:1214 level=info msg="Completed loading of configuration file" filename=prometheus.yml totalDuration=158.90387ms db_storage=1.513µs remote_storage=2.05µs web_handler=507ns query_engine=1.221µs scrape=135.715438ms scrape_sd=238.9µs notify=176.176µs notify_sd=87.698µs rules=4.778µs tracing=187.497µs
ts=2022-07-02T10:06:45.327Z caller=main.go:957 level=info msg="Server is ready to receive web requests."
ts=2022-07-02T10:06:45.327Z caller=manager.go:937 level=info component="rule manager" msg="Starting rule manager..."

#解压后当前目录会包含默认的Prometheus配置文件promethes.yml，同时prometheus的数据默认会存放在应用所在目录下data文件夹中
[root@host125 prometheus-2.36.0-rc0.linux-amd64]# ls
console_libraries  consoles  data  LICENSE  nohup.out  NOTICE  prometheus  prometheus.yml  promtool

在浏览器中访问部署的prometheus：http://192.168.220.125:9090，并切换到Status->Targets标签，可以看到node目标和mysql目标已启动
在这里插入图片描述

安装启动grafana

[root@host125 install]# tar -xvf grafana-enterprise-9.0.2.linux-amd64.tar.gz 
[root@host125 bin]# cd grafana-9.0.2/bin
[root@host125 bin]# nohup ./grafana-server web &

在浏览器中访问部署的grafana：http://192.168.220.125:3000 用户名/密码：admin/admin
在这里插入图片描述
添加一个数据源，选择prometheus，在url栏填入http://192.168.220.125:9090