K8S单机部署-11.安装Kubernetes Metrics Server监控

转载请注明出处

相关链接

• K8S单机部署-00.旧版本卸载
• K8S单机部署-01.单机部署K8S
• K8S单机部署-02.Kuboard面板部署
• K8S单机部署-03.验证K8S的第一个Deployment
• K8S单机部署-04.NFS服务器搭建
• K8S单机部署-05.静态持久化之PV、PVC(基于NFS)
• K8S单机部署-06.动态持久化StorageClass(基于NFS)
• K8S单机部署-07.调整nodePort端口范围
• K8S单机部署-08.安装ingress-nginx-controller
• K8S单机部署-10.增加自定义域名解析
• K8S单机部署-11.安装Kubernetes Metrics Server监控

现象

当需要查看资源的占用的时候执行以下命令， 提示缺少组件：

[root@master k8s-metric-server]# kubectl top pod
error: Metrics API not available

安装Metric-Server

版本关系

下载部署文件

参考 https://github.com/kubernetes-sigs/metrics-server

wget https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml -O metrics-server-components.yaml

若下载有问题可以直接复制下面内容 (以下为原始文件)

metrics-server-components.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    k8s-app: metrics-server
    rbac.authorization.k8s.io/aggregate-to-admin: "true"
    rbac.authorization.k8s.io/aggregate-to-edit: "true"
    rbac.authorization.k8s.io/aggregate-to-view: "true"
  name: system:aggregated-metrics-reader
rules:
- apiGroups:
  - metrics.k8s.io
  resources:
  - pods
  - nodes
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    k8s-app: metrics-server
  name: system:metrics-server
rules:
- apiGroups:
  - ""
  resources:
  - nodes/metrics
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - pods
  - nodes
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server-auth-reader
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: extension-apiserver-authentication-reader
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server:system:auth-delegator
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:auth-delegator
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: system:metrics-server
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:metrics-server
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
spec:
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    k8s-app: metrics-server
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
spec:
  selector:
    matchLabels:
      k8s-app: metrics-server
  strategy:
    rollingUpdate:
      maxUnavailable: 0
  template:
    metadata:
      labels:
        k8s-app: metrics-server
    spec:
      containers:
      - args:
        - --cert-dir=/tmp
        - --secure-port=10250
        - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
        - --kubelet-use-node-status-port
        - --metric-resolution=15s
        - --kubelet-insecure-tls
        image: registry.k8s.io/metrics-server/metrics-server:v0.7.0
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /livez
            port: https
            scheme: HTTPS
          periodSeconds: 10
        name: metrics-server
        ports:
        - containerPort: 10250
          name: https
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /readyz
            port: https
            scheme: HTTPS
          initialDelaySeconds: 20
          periodSeconds: 10
        resources:
          requests:
            cpu: 100m
            memory: 200Mi
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          runAsUser: 1000
          seccompProfile:
            type: RuntimeDefault
        volumeMounts:
        - mountPath: /tmp
          name: tmp-dir
      nodeSelector:
        kubernetes.io/os: linux
      priorityClassName: system-cluster-critical
      serviceAccountName: metrics-server
      volumes:
      - emptyDir: {}
        name: tmp-dir
---
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
  labels:
    k8s-app: metrics-server
  name: v1beta1.metrics.k8s.io
spec:
  group: metrics.k8s.io
  groupPriorityMinimum: 100
  insecureSkipTLSVerify: true
  service:
    name: metrics-server
    namespace: kube-system
  version: v1beta1
  versionPriority: 100

修改镜像地址

需要注意的是，metrics-server镜像用的是gcr镜像，需要换到国内的镜像仓库
将原始yaml文件中的

# 将 registry.k8s.io/metrics-server/metrics-server:v0.7.0

# 替换为以下镜像
registry.aliyuncs.com/google_containers/metrics-server:v0.7.0

修改TLS配置, 在原始metrics-server-components.yaml文件对应位置增加下面的参数, 具体原因在下面 问题一

......
      containers:
      - args:
        ...
        - --kubelet-insecure-tls
......

部署

kubectl apply -f metrics-server-components.yaml

验证效果

# 执行以下命令可以成功返回pod的资源使用情况
[root@master k8s-metric-server]# kubectl top pod

NAME                                        CPU(cores)   MEMORY(bytes)
mysql-5d7f8bd64f-7kcnx                      3m           661Mi
nfs-client-provisioner-867b699cb5-nvhmx     2m           8Mi
center-offline-5b66d879b8-79kk8             2m           827Mi
a-offline-5f68f9489-jl7lt                   5m           950Mi
b-offline-5f68f9489-xkc7p                   3m           1034Mi

问题一

Apply 之后查看pod状态, pod开始running, 但是一直不ready

[root@master k8s-metric-server]# kubectl get pods -n kube-system | grep metrics

metrics-server-7587785854-t8dpl   0/1     Running            0                  5m28s

原因

官方文档中, 如下红框中说明，Kubelet证书需要由群集证书颁发机构签名（或可以禁用证书验证，通过对Metrics Server配置参数–Kubelet-insecure-tls不安全）

解决办法

如下红框中说明，添加了“–Kubelet-insecure-tls”这个配置，就不会去验证Kubelets提供的服务证书的CA

我们对应修改原始yaml文件, 增加参数

......
      containers:
      - args:
        ...
        - --kubelet-insecure-tls
......

修改之后重新执行

kubectl apply -f metrics-server-components.yaml

重新查看状态, 可以发现已经ready

[root@master k8s-metric-server]# kubectl get pods -n kube-system | grep metrics
metrics-server-5cc466bcdc-vbjjb   1/1     Running            0                  62s