k8s/prometheus监控k8s集群

k8s使用

提示：记录k8s使用
第六章 kube-state-metrics监控k8s集群信息

一、说明

选型如下：（与之前文章相同）

名称	功能	版本
prometheus	采集数据	2.29.1
grafana	图表展示	8.1.1
kube-state-metrics	监控k8s容器	2.1.1

二、部署

prometheus与grafana部署见本系列第四章

1. RBAC权限

网上的各类教程好多都没说明要权限
kube-metric-state需要一定的RBAC权限，配置也简单。本文分开说明，实际操作中可修改相应名称(一般是namespace)后一次性执行下列几个yaml

创建ServiceAccount

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app: kube-state-metrics
  name: kube-state-metrics
  namespace: default

创建ClusterRole

一般这里不用改

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kube-state-metrics
rules:
- apiGroups: [""]
  resources:
  - configmaps
  - secrets
  - nodes
  - pods
  - services
  - resourcequotas
  - replicationcontrollers
  - limitranges
  - persistentvolumeclaims
  - persistentvolumes
  - namespaces
  - endpoints
  verbs: ["list", "watch"]
- apiGroups: ["extensions"]
  resources:
  - daemonsets
  - deployments
  - replicasets
  - ingresses
  verbs: ["list", "watch"]
- apiGroups: ["apps"]
  resources:
  - daemonsets
  - deployments
  - replicasets
  - statefulsets
  verbs: ["list", "watch"]
- apiGroups: ["batch"]
  resources:
  - cronjobs
  - jobs
  verbs: ["list", "watch"]
- apiGroups: ["autoscaling"]
  resources:
  - horizontalpodautoscalers
  verbs: ["list", "watch"]
- apiGroups: ["policy"]
  resources:
  - poddisruptionbudgets
  verbs: ["list", "watch"]
- apiGroups: ["certificates.k8s.io"]
  resources:
  - certificatesigningrequests
  verbs: ["list", "watch"]
- apiGroups: ["storage.k8s.io"]
  resources:
  - storageclasses
  verbs: ["list", "watch"]
- apiGroups: ["autoscaling.k8s.io"]
  resources:
  - verticalpodautoscalers
  verbs: ["list", "watch"]

绑定ClusterRole与ServiceAccount

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app: kube-state-metrics
  name: kube-state-metrics
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kube-state-metrics
subjects:
- kind: ServiceAccount
  name: kube-state-metrics
  namespace: default

三个配置可在一个kube-state-metrics.yaml文件中，之后执行

kubectl apply -f kube-state-metrics.yaml

检查

完成上述后，可在kuboard中进行查看。在“访问控制–第二阶段控制”
(1) ClusterRole中有创建的kube-state-metrics

(2) ServiceAccount中有创建的kube-state-metrics并且绑定了ClusterRole

2. kube-state-metrics

工作负载配置

（1）负载类型：部署，分层：监控
（2）容器信息添加端口，为了之后向外映射

（3）高级设置中添加ServiceAccount

（4）服务中添加端口映射

检查

做了端口映射，访问ip:8081

在healthz中有OK，在metrics里面有数据，则正常
一开始我按照网上教程做，没进行上述RBAC，则metrics里面全是#的注释没有数据

三、配置

prometheus

在配置字典中添加job（如果不清楚配置字典，可参考本系列第四章部署prometheus中）

  - job_name: "kube-state-metrics"
    static_configs:
      - targets: ["kube-state-metrics:8081"]
        labels:
          instance: 'node1'

重启prometheus容器，查看prometheus的targets

grafana

import dashboard模板13105、13332(后续再调整)

参考

监控k8s
kube-state-metrics