Prometheus+Grafana监控平台搭建

Prometheus+Grafana监控平台搭建

---

Prometheus安装

官网地址: https://prometheus.io/

下载prometheus和node_exporter

启动node_exporter

# nohup ./node_exporter &

安装Prometheus

编辑prometheus.yml, 将node_exporter添加到Prometheus目标对象,因为这里node_exporter和Prometheus安装在同一台机器,使用localhost即可，node_exporter端口9100

vim prometheus.yml
- job_name: 'localnode'
    static_configs:
      - targets: ['localhost:9100']

使用配置文件启动Prometheus

# nohup ./prometheus --config.file=prometheus.yml &

将Prometheus配置为系统服务

cd /usr/lib/systemd/system

创建文件：vim prometheus.service

[Unit]
Description=https://prometheus.io

[Service]
Restart=on-failure
ExecStart=/opt/prometheus/prometheus --config.file=/opt/prometheus/prometheus.yml --web.listen-address=:9090

[Install]                      
WantedBy=multi-user.target

生效系统system文件

systemctl daemon-reload

启动服务

systemctl start prometheus

客户端，配置服务发现监控linux主机及相关服务

node_exporter添加为服务方式启动

vim /usr/lib/systemd/system/node_exporter.service
[Unit]
Description=node_exporter
After=network.target 

[Service]
ExecStart=/opt/node_exporter/node_exporter
Restart=on-failure

[Install]
WantedBy=multi-user.target

生效系统system文件

systemctl daemon-reload

启动服务

systemctl start node_exporter

nginx，配置服务发现监控服务

下载nginx-vts-exporter

coscmd -b op-workerspace-1302719795 download nginx-vts-exporter-0.5.zip ./

nginx配置

# 主配置文件新增监控模块
vim /opt/nginx/conf/nginx.conf

vhost_traffic_status_zone;
vhost_traffic_status_filter_by_host on;

# 单独配置文件
vim metrics.conf
server
    {
        listen 100;
        server_name localhost;
        location  /metrics {
                vhost_traffic_status_display;
                vhost_traffic_status_display_format html;
        }
    }
    
# 检查nginx
/opt/nginx/sbin/nginx -t
# 重启nginx
systemctl reload nginx

nginx-vts-exporter添加为服务方式启动

vim /usr/lib/systemd/system/nginx_exporter.service
[Unit]
Description=nginx-vts-exporter
After=network.target 

[Service]
ExecStart=/opt/node_exporter/nginx/nginx-vts-exporter-0.5/bin/nginx-vts-exporter -nginx.scrape_uri=http://127.0.0.1:100/metrics/format/json
Restart=on-failure

[Install]
WantedBy=multi-user.target

生效系统system文件

systemctl daemon-reload

启动服务

systemctl start nginx_exporter

prometheus配置文件增加发现节点

vim prometheus.yml
# nginx指标
  - job_name: 'NGINX'
    scrape_interval: 15s
    static_configs:
      - targets: ['tx-bj5-web-dev-php-00:9913']
        labels:
          instance: 'dev-php-nginx'
      - targets: ['tx-bj5-web-dev-go-00:9913']
        labels:
          instance: 'dev-go-nginx'

Grafana Docker 镜像

运行Grafana 容器

启动容器绑定外部端口3000。

docker run -d --name=grafana -p 3000:3000 grafana/grafana

默认管理员用户是 admin/admin

smtp

#################################### SMTP / Emailing ##########################                       
[smtp]                                                                                                                  
enabled = true                                                                                                          
host = smtp.exmail.qq.com:465                                                                                           
user = developer@hsuanyuen.com                                                                                          
# If the password contains # or ; you have to wrap it with triple quotes. Ex """#password;"""                           
password = NXAPygimoTv6uvAc                                                                                             
;cert_file =                                                                                                            
;;key_file =                                                                                                            
skip_verify = true                                                                                                      
from_address = developer@hsuanyuen.com                                                                                  
from_name = Grafana                                                                                                     
;# EHLO identity in SMTP dialog (defaults to instance_name)                                                             
ehlo_identity = exmail.qq.com                                                                                           
;# SMTP startTLS policy (defaults to 'OpportunisticStartTLS')                                                           
;startTLS_policy = NoStartTLS

redis 插件

grafana-cli plugins install redis-datasource

node节点部署脚本

coscmd -b op-workerspace-1302719795 download node_exporter-1.2.2.linux-amd64.tar.gz ./
tar xzf node_exporter-1.2.2.linux-amd64.tar.gz
cat <<EOF | sudo tee /usr/lib/systemd/system/node_exporter.service
[Unit]
Description=node_exporter
After=network.target 

[Service]
ExecStart=/opt/node_exporter/node_exporter
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
mv node_exporter-1.2.2.linux-amd64 /opt/node_exporter
rm -rf node_exporter-1.2.2.linux-amd64.tar.gz
systemctl start node_exporter.service
netstat -nltp