kubeadm升级k8s集群

版本兼容性

1. K8s所有组件 kube-controller，kube-scheduler，kubelet的版本号不得高于kube-apiserver的版本号
2. 一些组件的版本号可低于kube-apiserver的1个次要版本
3. 在一个高可用集群中，多个kube-apiserver间的版本号最多只能相差一个次版本号
4. 最好所有组件与kube-apiserver版本号完全一致
5. 升级Kubernetes集群时，最先升级的核心组件就是kube-apiserver
6. 只能向上升级为一个次要版本
7. kubectl版本最多只能比kube-apiserver高或低一个次版本号

升级顺序

节点

1. 主控制节点
2. 子控制节点
3. Node节点

组件

1. kubeadm
2. 主控制节点各管理组件
3. 主控制节点的kubelet、kubectl
4. 子控制节点
5. Node节点
6. 检查验证

注意事项

1. 升级kubelet后需注意配置文件会被清空：/etc/sysconfig/kubelet
2. 由于版本的兼容性，只能从一个次要版本升级到另外一个次要版本，不能跳跃升级（没测试）
3. 在线升级Node节点前，尽量驱逐该Node上的应用

升级集群

现有集群

[root@pass1 ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T20:17:28Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T20:08:34Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}
[root@pass1 ~]# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T20:14:41Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}
[root@pass1 ~]# kubectl get nodes
NAME      STATUS    ROLES     AGE       VERSION
pass1     Ready     master    49d       v1.11.0
pass2     Ready     <none>    49d       v1.11.0
pass3     Ready     <none>    49d       v1.11.0

查看集群配置信息

[root@pass1 ~]# kubeadm config view > kubeadm.config
[root@pass1 ~]# kubeadm config view
api:
  advertiseAddress: 192.168.186.101
  bindPort: 6443
  controlPlaneEndpoint: ""
apiServerExtraArgs:
  authorization-mode: Node,RBAC
apiVersion: kubeadm.k8s.io/v1alpha2
auditPolicy:
  logDir: /var/log/kubernetes/audit
  logMaxAge: 2
  path: ""
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
etcd:
  local:
    dataDir: /var/lib/etcd
    image: ""
imageRepository: acaas_k8s/k8s.gcr.io   #这里为主要关注点
kind: MasterConfiguration
... ...

准备1.12.1的镜像和rpm包（本地已下载）

[root@pass1 ~]# ls k8s-1.12.1-image
coredns.tar  etcd.tar  kube-apiserver.tar  kube-controller-manager.tar  kube-proxy.tar  kube-scheduler.tar  load.sh
[root@pass1 ~]# ls rpm/
kubeadm-1.12.1-0.x86_64.rpm  kubectl-1.12.1-0.x86_64.rpm  kubelet-1.12.1-0.x86_64.rpm

安装rpm包同时加载镜像

[root@pass1 ~]# yum localinstall -y --disablerepo=* rpm/*.rpm
[root@pass1 ~]# sh k8s-1.12.1-image/load.sh

检查kubeadm、kubectl

[root@pass1 ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState:"clean", BuildDate:"2018-10-05T16:46:06Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T20:08:34Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}
[root@pass1 ~]# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState:"clean", BuildDate:"2018-10-05T16:43:08Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}
[root@pass1 ~]# cat /etc/sysconfig/kubelet 
KUBELET_EXTRA_ARGS=

重新配置kubelet

DOCKER_CGROUPS=$(sudo docker info | grep 'Cgroup' | cut -d' ' -f3)
tee /etc/sysconfig/kubelet<<EOF
KUBELET_EXTRA_ARGS="--feature-gates=AttachVolumeLimit=false --cgroup-driver=$DOCKER_CGROUPS --pod-infra-container-image=acaas_k8s/k8s.gcr.io/pause-amd64:3.1 --read-only-port=10255 --address=0.0.0.0 --root-dir=/data/kubelet"
EOF
systemctl daemon-reload 
systemctl restart kubelet

检查是否可更新版本

[root@pass1 ~]# kubeadm upgrade plan
[preflight] Running pre-flight checks.
[upgrade] Making sure the cluster is healthy:
[upgrade/config] Making sure the configuration is correct:
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[upgrade] Fetching available versions to upgrade to
[upgrade/versions] Cluster version: v1.11.0
[upgrade/versions] kubeadm version: v1.12.1
I0324 19:32:10.352804    6479 version.go:89] could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable.txt": Get https://storage.googleapis.com/kubernetes-release/release/stable.txt: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
I0324 19:32:10.352882    6479 version.go:94] falling back to the local client version: v1.12.1
[upgrade/versions] Latest stable version: v1.12.1
I0324 19:32:20.553189    6479 version.go:89] could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.11.txt": Get https://storage.googleapis.com/kubernetes-release/release/stable-1.11.txt: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
I0324 19:32:20.553222    6479 version.go:94] falling back to the local client version: v1.12.1
[upgrade/versions] Latest version in the v1.11 series: v1.12.1

Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':
COMPONENT   CURRENT       AVAILABLE
Kubelet     3 x v1.11.0   v1.12.1

Upgrade to the latest version in the v1.11 series:

COMPONENT            CURRENT   AVAILABLE
API Server           v1.11.0   v1.12.1
Controller Manager   v1.11.0   v1.12.1
Scheduler            v1.11.0   v1.12.1
Kube Proxy           v1.11.0   v1.12.1
CoreDNS              1.1.3     1.2.2
Etcd                 3.2.18    3.2.24

You can now apply the upgrade by executing the following command:

        kubeadm upgrade apply v1.12.1

_____________________________________________________________________

测试是否可进行更新

如下表示1.12.1版本已准备就绪，可以进行版本更新

[root@pass1 ~]# kubeadm upgrade apply v1.12.1 --config kubeadm.config --dry-run
... ...
[addons] Applied essential addon: kube-proxy
[dryrun] Finished dryrunning successfully!

升级集群

• 主控制节点

升级

[root@pass1 ~]# kubeadm upgrade apply v1.12.1 --config kubeadm.config
... ...

[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.12.1". Enjoy!

[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.

检查

[root@pass1 ~]# kubectl get nodes
NAME    STATUS   ROLES    AGE   VERSION
pass1   Ready    master   49d   v1.12.1
pass2   Ready    <none>   49d   v1.11.0
pass3   Ready    <none>   49d   v1.11.0

• Node节点

安装rpm包，加载镜像，重新配置kubelet后重启kubelet即可

[root@pass1 ~]# kubectl get nodes
NAME    STATUS   ROLES    AGE   VERSION
pass1   Ready    master   49d   v1.12.1
pass2   Ready    <none>   49d   v1.12.1
pass3   Ready    <none>   49d   v1.12.1