使用Prometheus来监控kubeadm部署的ETCD

环境说明：

1. 操作系统：Centos7-2009
2. 确保网络连接正常、yum源正常
3. k8s集群版本：1.21，集群安装为kubeadm，kube-prometheus-0.9.0
   

步骤说明：

1、查看ETCD是否可以暴露指标

curl --cacert /etc/kubernetes/pki/etcd/ca.crt --cert /etc/kubernetes/pki/etcd/healthcheck-client.crt --key /etc/kubernetes/pki/etcd/healthcheck-client.key https://192.168.10.30:2379/metrics

2、把ETCD的证书创建为secret

kubectl -n monitoring create secret generic etcd-certs --from-file=/etc/kubernetes/pki/etcd/ca.crt   --from-file=/etc/kubernetes/pki/etcd/healthcheck-client.crt  --from-file=/etc/kubernetes/pki/etcd/healthcheck-client.key

3、在prometheus里面引用这个secrets

kubectl -n monitoring edit prometheus k8s 
..........
spec:
.........
#结尾处
  secrets:
  - etcd-certs

4、prometheus会自动重启服务pod以加载这个secret配置，会自动重启pod配置，等待重启完成之后进去检查该容器的证书

kubectl -n monitoring exec -it prometheus-k8s-0 -c prometheus  -- sh 
/prometheus $ ls /etc/prometheus/secrets/etcd-certs/
ca.pem        etcd-key.pem  etcd.pem

5、创建service，并给port添加name

#创建SVC
kubectl expose pod -n kube-system etcd-ha-master1 --name=etcd-k8s --port=2379 --target-port=2379

#给名为etcd-k8s的svc，添加name的声明
kubectl -n kube-system edit svc etcd-k8s
spec:
.....
  ports:
  - name: api #添加name的声明
    port: 2379
    protocol: TCP
.........

6、创建ServiceMonitor，Etcd-ServiceMonitor.yaml

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: etcd-k8s
  namespace: monitoring
  labels:
    k8s-app: etcd-k8s
spec:
  jobLabel: k8s-app
  endpoints:
  - port: api
    interval: 30s
    scheme: https
    tlsConfig:
      caFile: /etc/prometheus/secrets/etcd-certs/ca.crt
      certFile: /etc/prometheus/secrets/etcd-certs/healthcheck-client.crt
      keyFile: /etc/prometheus/secrets/etcd-certs/healthcheck-client.key
      insecureSkipVerify: true
  selector:
    matchLabels:
      component: etcd
  namespaceSelector:
    matchNames:
    - kube-system

#应用Etcd-ServiceMonitor.yaml
 kubectl apply -f Etcd-ServiceMonitor.yaml

7、效果Prometheus的web界面效果