k8s之kubelet 配置证书轮换自签

• 在kubernetes1.7之后，可以采用集群自动签发证书方案，但仍然需要手动重启kubelet, 在1.8之后，就可以自动签发，自动renew证书；也可以设置更长的有效期。

• 方法1 .

• 查看证书有效期

root@node101 ~]#  kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Aug 06, 2022 14:43 UTC   354d                                    no      
apiserver                  Aug 06, 2022 14:43 UTC   354d            ca                      no      
apiserver-etcd-client      Aug 06, 2022 14:43 UTC   354d            etcd-ca                 no      
apiserver-kubelet-client   Aug 06, 2022 14:43 UTC   354d            ca                      no      
controller-manager.conf    Aug 06, 2022 14:43 UTC   354d                                    no      
etcd-healthcheck-client    Aug 06, 2022 14:43 UTC   354d            etcd-ca                 no      
etcd-peer                  Aug 06, 2022 14:43 UTC   354d            etcd-ca                 no      
etcd-server                Aug 06, 2022 14:43 UTC   354d            etcd-ca                 no      
front-proxy-client         Aug 06, 2022 14:43 UTC   354d            front-proxy-ca          no      
scheduler.conf             Aug 06, 2022 14:43 UTC   354d                                    no      

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Aug 04, 2031 14:43 UTC   9y              no      
etcd-ca                 Aug 04, 2031 14:43 UTC   9y              no      
front-proxy-ca          Aug 04, 2031 14:43 UTC   9y              no      

• 直接用一条命令更新（ kubeadm certs renew all ）

[root@node101 ~]# kubeadm  certs renew all 
[renew] Reading configuration from the cluster...
[renew] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed
certificate for serving the Kubernetes API renewed
certificate the apiserver uses to access etcd renewed
certificate for the API server to connect to kubelet renewed
certificate embedded in the kubeconfig file for the controller manager to use renewed
certificate for liveness probes to healthcheck etcd renewed
certificate for etcd nodes to communicate with each other renewed
certificate for serving etcd renewed
certificate for the front proxy client renewed
certificate embedded in the kubeconfig file for the scheduler manager to use renewed

Done renewing certificates. You must restart the kube-apiserver, kube-controller-manager, kube-scheduler and etcd, so that they can use the new certificates.

• 查看证书有效期

[root@node101 ~]# openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -text |grep ' Not '
            Not Before: Aug  6 14:43:36 2021 GMT
            Not After : Aug 16 15:48:32 2022 GMT


查看证书有效期
[root@node101 ~]# openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -text 
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1729213869226092457 (0x17ff66da2fe667a9)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=kubernetes
        Validity
            Not Before: Aug  6 14:43:36 2021 GMT
            Not After : Aug 16 15:48:32 2022 GMT
        Subject: CN=kube-apiserver
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)

[root@node101 ~]# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Aug 16, 2022 15:48 UTC   364d                                    no      
apiserver                  Aug 16, 2022 15:48 UTC   364d            ca                      no      
apiserver-etcd-client      Aug 16, 2022 15:48 UTC   364d            etcd-ca                 no      
apiserver-kubelet-client   Aug 16, 2022 15:48 UTC   364d            ca                      no      
controller-manager.conf    Aug 16, 2022 15:48 UTC   364d                                    no      
etcd-healthcheck-client    Aug 16, 2022 15:48 UTC   364d            etcd-ca                 no      
etcd-peer                  Aug 16, 2022 15:48 UTC   364d            etcd-ca                 no      
etcd-server                Aug 16, 2022 15:48 UTC   364d            etcd-ca                 no      
front-proxy-client         Aug 16, 2022 15:48 UTC   364d            front-proxy-ca          no      
scheduler.conf             Aug 16, 2022 15:48 UTC   364d                                    no      

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Aug 04, 2031 14:43 UTC   9y              no      
etcd-ca                 Aug 04, 2031 14:43 UTC   9y              no      
front-proxy-ca          Aug 04, 2031 14:43 UTC   9y              no      

• 配置证书轮换自签

vim /var/lib/kubelet/config.yaml

kind: KubeletConfiguration
serverTLSBootstrap: true   <<<<<---------添加这参数
logging: {}
memorySwap: {}
nodeStatusReportFrequency: 0s
nodeStatusUpdateFrequency: 0s
rotateCertificates: true
runtimeRequestTimeout: 0s
shutdownGracePeriod: 0s
shutdownGracePeriodCriticalPods: 0s
staticPodPath: /etc/kubernetes/manifests
streamingConnectionIdleTimeout: 0s
syncFrequency: 0s
volumeStatsAggPeriod: 0s

• 最初，来自节点上 kubelet 的证书签名请求处于 Pending 状态。 如果证书签名请求满足特定条件， 控制器管理器会自动批准，此时请求会处于 Approved 状态。 接下来，控制器管理器会签署证书， 证书的有效期限由 --cluster-signing-duration 参数指定，签署的证书会被附加到证书签名请求中。

• Kubelet 会从 Kubernetes API 取回签署的证书，并将其写入磁盘，存储位置通过 --cert-dir 参数指定。 然后 kubelet 会使用新的证书连接到 Kubernetes API。

• 当签署的证书即将到期时，kubelet 会使用 Kubernetes API，发起新的证书签名请求。 同样地，控制器管理器会自动批准证书请求，并将签署的证书附加到证书签名请求中。 Kubelet 会从 Kubernetes API 取回签署的证书，并将其写入磁盘。 然后它会更新与 Kubernetes API 的连接，使用新的证书重新连接到 Kubernetes API

• 获取到的csr为csr-cx5bt

[root@node101 ~]# kubectl get csr    
NAME        AGE   SIGNERNAME                      REQUESTOR             REQUESTEDDURATION   CONDITION
csr-cx5bt   14s   kubernetes.io/kubelet-serving   system:node:node101   <none>              Pending   <<<<-------状态暂时没有开启

执行并授权加入自签套餐

[root@node101 ~]# kubectl certificate approve csr-cx5bt
certificatesigningrequest.certificates.k8s.io/csr-cx5bt approved

查看生效

[root@node101 ~]# kubectl get csr
NAME        AGE   SIGNERNAME                      REQUESTOR             REQUESTEDDURATION   CONDITION
csr-cx5bt   53s   kubernetes.io/kubelet-serving   system:node:node101   <none>              Approved,Issued

更多参考

https://kubernetes.io/docs/tasks/tls/certificate-rotation/

https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/