k8s 老集群二进制 ca证书过期解决过程

如果有帮助到你，顺便点个赞哦～

问题现象：

执行kubectl get node 

报错：certificate has expired or is not yet valid

排查思路：

查看/etc/kubernetes/ssl/ca.pem 有效期

[root@c3-a10-136-120-15-c1-64 ~]# openssl x509 -noout -dates -in /etc/kubernetes/ssl/ca.pem 
notBefore=Feb 28 04:07:30 2023 GMT
notAfter=Feb 27 04:07:30 2023 GMT

查看kube-apiserver日志

journalctl -n 2000 -fu kube-apiserver 

可以确定确实是ca跟证书过期了

解决方案：

1. 备份原来的配置文件及证书

cp -r /etc/kubernetes /home/sunwenbo/kubernetes-old-2023-2-28
cp -r /root/.kube /home/sunwenbo/.kube-old-2023-2-28

2. 重新颁发ca证书

生成证书请求文件csr，需要使用原ca.pem 证书和ca-key.pem 公钥
openssl x509 -x509toreq -in ca.pem -signkey ca-key.pem -out new-server.csr 

根据csr证书请求文件重新生成ca证书
openssl x509 -req -days 3650 -in new-server.csr -signkey ca-key.pem -out new-ca.pem 

进行测试验证,返回OK则代表正常
openssl verify -CAfile new-ca.pem -verbose /etc/etcd/ssl/etcd.pem 
/etc/etcd/ssl/etcd.pem: OK

替换证书
mv new-ca.pem /etc/kubernetes/ssl/ca.pem

3. 更新.kube/config 文件（如果有多个master的话，每个节点都需要替换）

得到ca.pem证书base64编码后的内容

cat -n /etc/kubernetes/ssl/ca.pem | base64 

修改.kube/config文件中的certificate-authority-data 字段内容

4.  重启etcd和kube-apiserver （必须重启）

systemctl  restart etcd.service
systemctl  restart kube-apiserver.service

5. 验证

查看etcd集群状态

[root@c3-a10-136-120-15-c1-64 ssl]# etcdctl --endpoint=https://10.136.121.22:2379 --ca-file=/etc/kubernetes/ssl/ca.pem --key-file=/etc/etcd/ssl/etcd-key.pem --cert-file=/etc/etcd/ssl/etcd.pem cluster-health 
member 7e41471da948b50 is healthy: got healthy result from https://10.136.120.15:2379
member 2d7579278801c0e8 is healthy: got healthy result from https://10.136.122.15:2379
member dfb02b9a52d3e722 is healthy: got healthy result from https://10.136.121.22:2379
cluster is healthy

[root@c3-a10-136-120-15-c1-64 ssl]# kubectl get node 
NAME            STATUS     ROLES    AGE      VERSION
10.136.123.17   Ready      <none>   3y289d   v1.11.3-custom.1
10.136.123.18   Ready      <none>   3y289d   v1.13.5-custom.0
10.136.125.5    Ready      <none>   2y337d   v1.11.3-custom.1
10.136.125.6    Ready      <none>   2y337d   v1.11.3-custom.1
10.136.125.9    Ready      <none>   2y331d   v1.11.3-custom.1
10.136.126.20   Ready      <none>   2y346d   v1.11.3-custom.1
10.136.14.11    Ready      <none>   3y305d   v1.13.5-custom.0
10.136.142.23   Ready      <none>   3y289d   v1.13.5-custom.0
10.136.145.11   Ready      <none>   3y289d   v1.11.3-custom.1
10.136.145.13   Ready      <none>   3y273d   v1.13.5-custom.0
10.136.145.14   Ready      <none>   3y273d   v1.13.5-custom.0
10.136.145.19   Ready      <none>   3y273d   v1.13.5-custom.0