kubernetes 踩坑记录

前言

这几周一直在玩k8s 1.30版本，想着尝鲜一下，发现与之前1.22版本搭建的环境大不相同，索性记录一下踩过的坑

环境

• kubeadm
• kubernetes 1.30
• kubelet\kubectl
• containerd 1.7.19（runc、CNI插件）
• flannel（包含containerd for CNI插件 ）

配置google镜像

老生常谈的问题了，直接甩命令

kubeadm init --v=5 --pod-network-cidr=10.244.0.0/16 --image-repository=registry.aliyuncs.com/google_containers

root用户操作kubeadm init

请注意，init后给了一个指令推荐，让我们执行如下指令，如果不执行会无法使用kubectl

export KUBECONFIG=/etc/kubernetes/admin.conf
echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile
source ~/.bash_profile

打开linux文件最大句柄

sudo vim /etc/security/limits.conf
* soft nofile 65535
* hard nofile 65535
# 检查是否生效
ulimit -n

关闭swap

sudo vim /etc/fstab
# 查找包含 swap 的行，并在该行前添加 # 进行注释。例如
#/swapfile swap swap defaults 0 0
# 检查是否生效
sudo swapon --show

设置containerd镜像

vim /etc/containerd/config.toml
# in config.toml file
version = 2

[plugins]
  [plugins."io.containerd.grpc.v1.cri"]
    sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.8"
    [plugins."io.containerd.grpc.v1.cri".registry]
      [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
          endpoint = ["https://registry.aliyuncs.com"]

很重要的警告

请勿将生成的admin.conf分享给其他人，这样会导致严重的安全问题

基本的节点如下

NAMESPACE      NAME                                            READY   STATUS    RESTARTS   AGE
kube-flannel   kube-flannel-ds-mx79b                           1/1     Running   0          2m39s
kube-system    coredns-7b5944fdcf-6lmst                        1/1     Running   0          5m2s
kube-system    coredns-7b5944fdcf-zn624                        1/1     Running   0          5m2s
kube-system    etcd-localhost.localdomain                      1/1     Running   2          5m17s
kube-system    kube-apiserver-localhost.localdomain            1/1     Running   2          5m17s
kube-system    kube-controller-manager-localhost.localdomain   1/1     Running   0          5m18s
kube-system    kube-proxy-56xv5                                1/1     Running   0          5m2s
kube-system    kube-scheduler-localhost.localdomain            1/1     Running   2          5m17s

手动获取discovery-token-ca-cert-hash

openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | \
   openssl dgst -sha256 -hex | sed 's/^.* //'
# 返回hash值
# 8cb2de97839780a412b93877f8507ad6c94f73add17d5d7058e91741c9d5ec78

加入工作节点注意集群内名称唯一

需要修改两个文件/etc/hostname和/etc/hosts，如果名称一样无法加入节点，注意修改完成后需要重启服务器，随后加入控制端就可以了

# 加入节点的join语句在init控制端需要保存
kubeadm join --v=5 192.168.31.155:6443 --token e9wx42.7hbclvkswuok2c56 \
--discovery-token-ca-cert-hash sha256:c9da8c543b34a301e018441f34294664c9345582c55d9799d07e81b8f490593b
# 会显示如下提示
  to the Node API object "localhost.localdomain158" as an annotation

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

工作节点无法使用kubectl(也许设计的就是只使用控制面板节点)

# 网上找到的答案一般都是如下，先按照这个方式解决吧
scp root@192.168.31.155:/etc/kubernetes/admin.conf /etc/kubernetes/
export KUBECONFIG=/etc/kubernetes/admin.conf
echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile
source ~/.bash_profile

提供更高的CoreDNS 可用性

kubectl -n kube-system rollout restart deployment coredns

结尾

最后就搭建了一个小的k8s集群

kubectl get nodes
# 显示如下
NAME                       STATUS   ROLES           AGE    VERSION
localhost.localdomain      Ready    control-plane   148m   v1.30.2
localhost.localdomain156   Ready    <none>          19m    v1.30.2
localhost.localdomain157   Ready    <none>          19m    v1.30.2
localhost.localdomain158   Ready    <none>          19m    v1.30.2