监控不携带metrics接口的服务(nginx、mysql、redis)

prometheus的架构图

在这里插入图片描述

监控不携带metrics接口的服务

普罗米修斯的两种监控服务:

​ 1.携带metrics接口的服务

​ 可以通过metrics接口来获取服务的监控项、监控信息

​ 2.不携带metrics接口的服务

​ 通过`EndPoint`造一个metrict接口

​ PS:metrics(接口)

监控流程:

1、部署exporter,从而创建一个metrics接口

2、部署EndPoints,链接expertor暴露出来的metrics接口

3、部署Service,基于ServiceMonitor使用

4、创建ServiceMonitor,注入promethues

5、测试服务

6、加入grafana,做大屏展示

一、监控nginx

0 、准备测试环境

server {
    listen       80;
    server_name  localhost;

    location /status {
        stub_status;
    }


在这里插入图片描述

Active connections: 当前nginx正在处理的活动连接数.
 
Server accepts handled requests: 
    nginx启动以来总共处理了23个连接
    成功创建23握手(证明中间没有失败的)
    总共处理了177个请求。
 
Reading: nginx读取到客户端的Header信息数.
Writing: nginx返回给客户端的Header信息数.
Waiting: 开启keep-alive的情况下,这个值等于 active – (reading + writing),意思就是nginx已经处理完成,正在等候下一次请求指令的驻留连接。
所以,在访问效率高,请求很快被处理完毕的情况下,Waiting数比较多是正常的.如果reading +writing数较多,则说明并发访问量非常大,正在处理过程中。

1、部署expertor,从而创建一个`metrics接口

`nginx-prometheus-exporter镜像地址: `https://github.com/nginxinc/nginx-prometheus-exporter

依据这个命令造作
$ docker run -p 9113:9113 nginx/nginx-prometheus-exporter:0.9.0 -nginx.plus -nginx.scrape-uri=http://<nginx-plus>:8080/api


端口:9113

在这里插入图片描述

  • 部署expertor,从而创建一个metrics接口

1)准备目录
[root@k8s-master1 ~]# mkdir exporter-nginx
[root@k8s-master1 ~]# cd exporter-nginx/
[root@k8s-master1 exporter-nginx]# vim exporter-nginx-deployment.yaml


2)创建服务
创建一个服务,获取并格式化metrics接口数据

cat > exporter-nginx-deployment.yaml <<EOF
kind: Deployment
apiVersion: apps/v1
metadata:
  name: nginx-prometheus-exporter
spec:
  selector:
    matchLabels:
      app: nginx-prometheus-exporter
      deploy: nginx-exporter
  template:
    metadata:
      labels:
        app: nginx-prometheus-exporter
        deploy: nginx-exporter
    spec:
      containers:
        - name: nginx-exporter
          image: nginx/nginx-prometheus-exporter:0.9.0
          imagePullPolicy: IfNotPresent
          command:
            - "nginx-prometheus-exporter"
            - "-nginx.scrape-uri=http://192.168.15.32/status"   #因为提前已经在192.168.15.32上安装了nginx   ,所以这里填写nginx安装的地址ip
EOF


3)部署服务
[root@k8s-master1 exporter-nginx]# kubectl apply -f exporter-nginx-deployment.yaml 
deployment.apps/name-prometheus-exporter created

  • 创建一个Service,给予prometheus集群获取格式化好了的metrics接口服务
cat > exporter-service.yaml EOF
kind: Service
apiVersion: v1
metadata:
  name: nginx-prometheus-exporter
spec:
  ports:
    - port: 9113          #端口9113
      targetPort: 9113    #容器内部也是9113
      name: nginx-prometheus-exporter
      protocol: TCP
  selector:
    app: nginx-prometheus-exporter
    deploy: nginx-exporter
EOF



[root@k8s-master1 exporter-nginx]# kubectl apply -f exporter-service.yaml


6)查看部署
[root@k8s-master-01 nginx]# kubectl get pods,svc
NAME  READY   STATUS   RESTARTS   AGE
pod/nginx-prometheus-exporter-776f76467-59msl            1/1     Running            0          5m55s

NAME TYPE CLUSTER-IP  EXTERNAL-IP   PORT(S)          AGE
service/nginx-prometheus-exporter   ClusterIP   10.110.27.71    <none>        9113/TCP         2m50s



7)访问测试
[root@k8s-master-01 nginx]# curl 10.110.27.71:9113/metrics
# HELP nginx_connections_accepted Accepted client connections
# TYPE nginx_connections_accepted counter
nginx_connections_accepted 4
# HELP nginx_connections_active Active client connections
# TYPE nginx_connections_active gauge
nginx_connections_active 1
# HELP nginx_connections_handled Handled client connections
# TYPE nginx_connections_handled counter

这就意味着nginx有这个指标了

2、部署Endpoints,链接expertor暴露出来的metrics接口

# 参考上述Service,此处无需重复创建。

[root@k8s-master-01 nginx]# kubectl get endpoints
NAME       ENDPOINTS            AGE
nginx-prometheus-exporter   10.244.1.128:9113    21m

[root@k8s-master-01 nginx]# curl  10.244.1.128:9113/metrics
...
# HELP nginx_http_requests_total Total http requests
# TYPE nginx_http_requests_total counter
nginx_http_requests_total 5
# HELP nginx_up Status of the last metric scrape
# TYPE nginx_up gauge
nginx_up 1
# HELP nginxexporter_build_info Exporter build information
# TYPE nginxexporter_build_info gauge
nginxexporter_build_info{commit="5f88afbd906baae02edfbab4f5715e06d88538a0",date="2021-03-22T20:16:09Z",version="0.9.0"} 1
....


理论上建立一个service就会对应建立一个endpoints

3、部署Service,基于ServiceMonitor使用

# 参考上述Service,此处无需重复创建。

cat > exporter-service.yaml EOF
kind: Service
apiVersion: v1
metadata:
  name: nginx-prometheus-exporter
spec:
  ports:
    - port: 9113          #端口9113
      targetPort: 9113    #容器内部也是9113
      name: nginx-prometheus-exporter
      protocol: TCP
  selector:
    app: nginx-prometheus-exporter
    deploy: nginx-exporter
EOF

4、创建ServiceMonitor,注入promethues

cat > service-monitor.yaml <<EOF
kind: ServiceMonitor
apiVersion: monitoring.coreos.com/v1
metadata:
  labels:   #选择service
    app: nginx-prometheus-exporter-servicemonitor
    deploy: nginx-exporter-servicemonitor
  name: nginx-monitor
  namespace: monitoring    #普罗米修斯的命名空间
spec:
  endpoints:
    - interval: 3s
      port: nginx-prometheus-exporter   #用端口或者名字都可以
  selector:
    matchLabels:
      app: nginx-prometheus-exporter
      deploy: nginx-exporter
  namespaceSelector:
    matchNames:
      - "default"
EOF



2)部署查看
[root@k8s-master1 exporter-nginx]# kubectl apply -f service-monitor.yaml
[root@k8s-master1 exporter-nginx]# kubectl get servicemonitor -n monitoring 
NAME                      AGE
nginx-monitor             25s


3)测试(查看prometheus的控制器)
[root@k8s-master-01 nginx]# kubectl get prometheus -n monitoring 
NAME   VERSION   REPLICAS   AGE
k8s    v2.15.2   2          29h

删除重新部署一下
[root@k8s-master-01 opt]# kubectl delete prometheus -n monitoring k8s 
prometheus.monitoring.coreos.com "k8s" deleted

prometheus-k8s.yaml (见附件)
[root@k8s-master-01 opt]# kubectl apply -f prometheus-k8s.yaml 



重启普罗米修斯监控pod(prometheus-k8s-0),载入监控项(见附件)


4)加入grafana,做大屏展示
略~


grafana官网:https://grafana.com/grafana/dashboards/

在这里插入图片描述

附件:

kind: Prometheus
apiVersion: monitoring.coreos.com/v1
metadata:
  labels:
    prometheus: k8s
  name: k8s
  namespace: monitoring
spec:
  alerting:
    alertmanagers:
      - name: alertmanager-main
        namespace: monitoring
        port: web
      - name: alertmanager-main-etcd
        namespace: kube-system
        port: etcd
  image: quay.io/prometheus/prometheus:v2.15.2
  nodeSelector:
    kubernetes.io/os: linux
  podMonitorNamespaceSelector: {}
  podMonitorSelector: {}
  replicas: 2
  resources:
    requests:
      memory: 400Mi
  ruleSelector:
    matchLabels:
      prometheus: k8s
      role: alert-rules
  securityContext:
    fsGroup: 2000
    runAsNonRoot: true
    runAsUser: 1000
  serviceAccountName: prometheus-k8s
  serviceMonitorNamespaceSelector: {}
  serviceMonitorSelector: {}
  version: v2.15.2
  secrets:
    - etcd-certs
    

二、监控redis

https://github.com/oliver006/redis_exporter

在这里插入图片描述
在这里插入图片描述

1.部署测试环境

1)编写部署清单

cat > redis-deploy.yaml <EOF
kind: Deployment
apiVersion: apps/v1
metadata:
  name: redis
spec:
  selector:
    matchLabels:
      k8s: redis-exporter
  template:
    metadata:
      labels:
        k8s: redis-exporter
    spec:
      containers:
        - name: redis
          imagePullPolicy: IfNotPresent
          image: redis
EOF

2)编写Service清单

cat > redis-svc.yaml <<EOF
kind: Service
apiVersion: v1
metadata:
  name: redis-svc
  labels:
    k8s: redis-exporter
spec:
  ports:
    - port: 6379
      targetPort: 6379
      name: redis
  selector:
    k8s: redis-exporter
EOF

3)部署

[root@k8s-master1 exporter-redis]# kubectl apply -f redis-deploy.yaml
[root@k8s-master1 exporter-redis]# kubectl apply -f redis-svc.yaml

2.配置部署exporter

  • 配置部署exporter,从而创建一个metrics接口

1)编写部署清单

cat > redis-exporter-deploy.yaml <<EOF
kind: Deployment
apiVersion: apps/v1
metadata:
  name: redis-exporter-server
spec:
  selector:
    matchLabels:
      k8s: redis-exporter-server
  template:
    metadata:
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "9121"
      labels:
        k8s: redis-exporter-server
    spec:
      containers:
        - name: redis
          image: oliver006/redis_exporter
          args:
            - "--redis.addr=redis://redis-svc.default.svc.cluster.local:6379"
          ports:
            - containerPort: 9121
EOF

2)编写svc清单

cat > redis-exporter-svc.yaml <<EOF
kind: Service
apiVersion: v1
metadata:
  name: redis-exporter-server
  labels:
    k8s: redis-exporter-server-svc
spec:
  ports:
    - port: 9121
      targetPort: 9121
      name: redis-exporter
  selector:
    k8s: redis-exporter-server
EOF

3)部署

[root@k8s-master1 exporter-redis]# kubectl apply -f redis-exporter-deploy.yaml

[root@k8s-master-01 redis]# kubectl get pods
NAME                                                 READY   STATUS              RESTARTS   AGE
redis-6b9dd9c556-7p2b2                               1/1     Running             0          4m54s
redis-exporter-server-6f6476f86f-b7kqk               1/1     Running             0          4m1s
    

[root@k8s-master1 exporter-redis]# kubectl apply -f redis-exporter-svc.yaml


[root@k8s-master-01 redis]# kubectl get svc
NAME                        TYPE        CLUSTER-IP      
redis-svc                   ClusterIP   10.102.221.62   <none>        6379/TCP         105m

3.部署Endpoints

  • 部署Endpoints,链接exporter暴露出来的metrics接口
# 参考上述Service,此处无需重复创建。

4.部署Service

  • 部署Service,基于ServiceMonitor使用
# 参考上述Service,此处无需重复创建。

5.创建ServiceMonitor清单

  • 创建ServiceMonitor,注入promethues

1)编写配置清单

cat > redis-monitor.yaml <<EOF
kind: ServiceMonitor
apiVersion: monitoring.coreos.com/v1
metadata:
  labels:
    app: redis-prometheus-exporter-servicemonitor
    deploy: redis-exporter-servicemonitor
  name: redis-monitor
  namespace: monitoring
spec:
  endpoints:
    - interval: 3s
      port: redis-exporter
  selector:
    matchLabels:
      k8s: redis-exporter-server-svc
  namespaceSelector:
    matchNames:
      - "default"
EOF

2)部署

[root@k8s-master1 exporter-redis]# kubectl apply -f redis-monitor.yaml

redis-monitor             2m

6.grafana展示

在这里插入图片描述

三、 监控数据库

1.造环境

1)安装mariadb

[root@k8s-master1]# yum install mariadb\* -y
[root@k8s-master1]# systemctl enable --now mariadb

2)授权monitor用户

[root@k8s-master1]# mysql
MariaDB [(none)]> grant select,replication client,process ON *.* to 'mysql_monitor'@'%' identified by 'oldboy@123';
Query OK, 0 rows affected (0.00 sec)

PS:授权ip为localhost,因为不是prometheus服务器来直接找mariadb 获取数据,而是prometheus服务器找mysql_exporter,mysql_exporter 再找mariadb,所以这个localhost是指的mysql_exporter的IP

2.配置maridb

创建一个mariadb配置文件,写上连接的用户名与密码(和上面的授权的用户名 和密码要对应)

[root@k8s-master1]# vim /usr/local/mysqld_exporter-0.12.1/.my.cnf
[client]
user=mysql_monitor
password=123

3.启动mysqld_exporter

[root@k8s-master1]# nohup /usr/local/mysqld_exporter-0.12.1/mysqld_exporter --config.my-cnf=/usr/local/mysqld_exporter-0.12.1/.my.cnf &

PS:确认端口是否为 9104

2.安装数据库exporter

1)编写部署清单

cat > mysql-deploy.yaml <<EOF
kind: Deployment
apiVersion: apps/v1
metadata:
  name: mysql-exporter-server
spec:
  selector:
    matchLabels:
      k8s: mysql-exporter-server
  template:
    metadata:
      labels:
        k8s: mysql-exporter-server
    spec:
      containers:
        - name: mysql
          image: prom/mysqld-exporter
          env:
            - name: DATA_SOURCE_NAME
              value: "mysql_monitor:oldboy@123@(mysql.default.svc.cluster.local:3306)"
          ports:
            - containerPort: 9104
EOF

2)编写Service清单

cat > mysql-svc.yaml <<EOF
kind: Service
apiVersion: v1
metadata:
  name: mysql-exporter-server
  labels:
    k8s: mysql-exporter-server-svc
spec:
  ports:
    - port: 9104
      targetPort: 9104
      name: mysql-exporter-port
      protocol: TCP
  selector:
    k8s: mysql-exporter-server
---
kind: ServiceMonitor
apiVersion: monitoring.coreos.com/v1
metadata:
  labels:
    app: mysql-prometheus-exporter-servicemonitor
    deploy: mysql-exporter-servicemonitor
  name: mysql-exporter-monitor
  namespace: monitoring
spec:
  endpoints:
    - interval: 3s
      port: mysql-exporter-port
  selector:
    matchLabels:
      k8s: mysql-exporter-server-svc
  namespaceSelector:
    matchNames:
      - "default"
EOF

3.部署服务

[root@k8s-master1 exporter-mysql]# kubectl apply -f mysql-deploy.yaml
[root@k8s-master1 exporter-mysql]# kubectl apply -f mysql-svc.yaml

监控exporter

  • 监控一个外部的exporter

1.造环境

1)下载部署exporter

#下载
[root@k8s-master1]# wget https://github.com/prometheus/mysqld_exporter/releases/download/v0.12.1/mysqld_exporter-0.12.1.linux-amd64.tar.gz

# 解压
[root@k8s-master1]# tar -xf mysqld_exporter-0.12.1.linux-amd64.tar.gz -C /usr/local/

#查看
[root@k8s-master1]# ll /usr/localmysqld_exporter-0.12.1.linux-amd64
总用量 14484
-rw-r--r-- 1 3434 3434    11325 729 2019 LICENSE
-rwxr-xr-x 1 3434 3434 14813452 729 2019 mysqld_exporter
-rw-r--r-- 1 3434 3434       65 729 2019 NOTICE

  • 1
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值