kubernetes集群编排——k8s 集群部署

k8s 集群部署

官网：

https://v1-23.docs.kubernetes.io/zh/docs/setup/production-environment/tools/kubeadm/install-kubeadm/

主机名	ip	角色
k8s1	192.168.52.130	reg.westos.org，harbor仓库
k8s2	192.168.52.131	master，k8s集群控制节点
k8s3	192.168.52.132	node，k8s集群工作节点
k8s4	192.168.52.133	node，k8s集群工作节点

所有节点禁用selinux和防火墙

所有节点同步时间和解析

所有节点安装docker-ce

所有节点禁用swap，注意注释掉/etc/fstab文件中的定义

集群环境初始化

所有k8s集群节点执行以下步骤

[root@k8s2 ~]# swapoff -a

[root@k8s2 ~]# vim /etc/fstab

#/dev/mapper/rhel-swap   swap                    swap    defaults        0 0

修改内核参数

[root@k8s2 sysctl.d]# vim docker.conf

net.bridge.bridge-nf-call-iptables=1

net.bridge.bridge-nf-call-ip6tables=1

net.ipv4.ip_forward=1

[root@k8s2 ~]#  sysctl --system

[root@k8s2 yum.repos.d]# vim docker.repo 

​
[docker]

name=docker-ce

baseurl=https://mirrors.tuna.tsinghua.edu.cn/docker-ce/linux/centos/7/x86_64/stable/

gpgcheck=0


[centos]

name=extras

baseurl=https://mirrors.tuna.tsinghua.edu.cn/centos/7/extras/x86_64/

gpgcheck=0

​

[root@k8s2 ~]# yum install -y docker-ce

[root@k8s2 ~]# systemctl enable --now docker

[root@k8s2 ~]# vim /etc/docker/daemon.json

{

        "registry-mirrors": ["https://reg.westos.org"],

        "exec-opts": ["native.cgroupdriver=systemd"],

        "log-driver": "json-file",

        "log-opts": {

                "max-size": "100m"

        },

        "storage-driver": "overlay2"

}

[root@k8s2 ~]# systemctl  restart docker

所有节点同步docker配置，以及拷贝harbor仓库的证书

[root@k8s1 ~]# cd /etc/docker/

[root@k8s1 docker]# ls

certs.d

[root@k8s1 docker]# scp -r certs.d/ k8s2:/etc/docker/

确保所有k8s节点可以从私有仓库下载镜像

[root@k8s2 docker]# docker pull nginx

所有节点安装kubeadm

[root@k8s2 yum.repos.d]# vim k8s.repo

​
[kubernetes]

name=Kubernetes

baseurl=https://mirrors.tuna.tsinghua.edu.cn/kubernetes/yum/repos/kubernetes-el7-x86_64/

gpgcheck=0

​

[root@k8s2 ~]# yum install -y kubelet-1.23.17-0 kubeadm-1.23.17-0 kubectl-1.23.17-0

[root@k8s2 ~]# systemctl enable --now kubelet

拉取集群所需镜像

[root@k8s2 ~]# kubeadm config images pull --image-repository registry.aliyuncs.com/google_containers

登录仓库

[root@k8s2 ~]# docker login reg.westos.org

Username: admin

Password:

先在harbor仓库上新建一个项目

上传镜像

[root@k8s2 ~]# docker images |grep google_containers | awk '{print $1":"$2}' | awk -F/ '{system("docker tag "$0" reg.westos.org/k8s/"$3"")}'

[root@k8s2 ~]# docker images  |grep k8s | awk '{system("docker push "$1":"$2"")}'

集群初始化

[root@k8s2 ~]# kubeadm init --pod-network-cidr=10.244.0.0/16 --image-repository reg.westos.org/k8s --kubernetes-version v1.23.17

设置环境变量

[root@k8s2 ~]# export KUBECONFIG=/etc/kubernetes/admin.conf

注意没有设置变量会有以下报错

写入环境变量文件，确保重启后依然生效

[root@k8s2 ~]# vim .bash_profile

export KUBECONFIG=/etc/kubernetes/admin.conf

查看集群状态

[root@k8s2 ~]# kubectl get node

NAME   STATUS     ROLES                  AGE   VERSION

k8s2   NotReady   control-plane,master   74s   v1.23.17

当前节点还没有就绪，是因为没有安装网路插件，pod还没运行

[root@k8s2 ~]# kubectl get pod -A

NAMESPACE     NAME                           READY   STATUS    RESTARTS   AGE

kube-system   coredns-7b56f6bc55-b495q       0/1     Pending   0          79s

kube-system   coredns-7b56f6bc55-ch2ts       0/1     Pending   0          79s

kube-system   etcd-k8s2                      1/1     Running   0          92s

kube-system   kube-apiserver-k8s2            1/1     Running   0          92s

kube-system   kube-controller-manager-k8s2   1/1     Running   0          92s

kube-system   kube-proxy-7ckfn               1/1     Running   0          79s

kube-system   kube-scheduler-k8s2            1/1     Running   0          92s

安装flannel网络插件

下载flannel网络插件

[root@k8s2 ~]# wget https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

修改镜像位置

[root@k8s2 ~]# vim kube-flannel.yml

新建项目仓库

下载镜像

[root@k8s1 docker]# docker pull docker.io/flannel/flannel:v0.21.2

[root@k8s1 docker]# docker pull docker.io/flannel/flannel-cni-plugin:v1.1.2

上传镜像

[root@k8s1 docker]# docker images |grep flannel | awk '{print $1":"$2}' | awk '{system("docker tag "$0" reg.westos.org/"$0"")}'

[root@k8s1 docker]# docker push reg.westos.org/flannel/flannel:v0.21.2

[root@k8s1 docker]# docker push reg.westos.org/flannel/flannel-cni-plugin:v1.1.2

确保镜像上传成功

部署网络插件

[root@k8s2 ~]# kubectl apply -f kube-flannel.yml

namespace/kube-flannel created

serviceaccount/flannel created

clusterrole.rbac.authorization.k8s.io/flannel created

clusterrolebinding.rbac.authorization.k8s.io/flannel created

configmap/kube-flannel-cfg created

daemonset.apps/kube-flannel-ds created

[root@k8s2 ~]# kubectl  -n kube-flannel get pod

NAME                    READY   STATUS    RESTARTS   AGE

kube-flannel-ds-6gnh4   1/1     Running   0          11s

[root@k8s2 ~]# kubectl get node

NAME   STATUS   ROLES                  AGE   VERSION

k8s2   Ready    control-plane,master   14m   v1.23.17

[root@k8s2 ~]# kubectl get pod -A

NAMESPACE      NAME                           READY   STATUS    RESTARTS   AGE

kube-flannel   kube-flannel-ds-6gnh4          1/1     Running   0          20s

kube-system    coredns-7b56f6bc55-b495q       1/1     Running   0          14m

kube-system    coredns-7b56f6bc55-ch2ts       1/1     Running   0          14m

kube-system    etcd-k8s2                      1/1     Running   0          14m

kube-system    kube-apiserver-k8s2            1/1     Running   0          14m

kube-system    kube-controller-manager-k8s2   1/1     Running   0          14m

kube-system    kube-proxy-7ckfn               1/1     Running   0          14m

kube-system    kube-scheduler-k8s2            1/1     Running   0          14m

扩容节点

[root@k8s3 ~]# kubeadm join 192.168.52.131:6443 --token od1a2o.lpdakq610m323uzl         --discovery-token-ca-cert-hash sha256:bebfed78424aa275e8061bbdf97a9e7b238d9a7f45b5f373186754db3114b1ec

[root@k8s4 ~]# kubeadm join 192.168.52.131:6443 --token od1a2o.lpdakq610m323uzl         --discovery-token-ca-cert-hash sha256:bebfed78424aa275e8061bbdf97a9e7b238d9a7f45b5f373186754db3114b1ec

[root@k8s2 ~]# kubectl get node

NAME   STATUS   ROLES                  AGE   VERSION

k8s2   Ready    control-plane,master   16m   v1.23.17

k8s3   Ready    <none>                 52s   v1.23.17

k8s4   Ready    <none>                 46s   v1.23.17

设置kubectl命令补齐

[root@k8s2 ~]# yum install -y bash-completion

[root@k8s2 ~]# echo "source <(kubectl completion bash)" >> ~/.bashrc

[root@k8s2 ~]# source  ~/.bashrc