kubernetes v1.18.6 高可用集群集群安装说明

最新推荐文章于 2023-03-20 00:08:48 发布

置顶 chengli.zou

最新推荐文章于 2023-03-20 00:08:48 发布

阅读量670

点赞数 1

分类专栏： shell k8s keepalived 文章标签： docker centos 负载均衡

本文链接：https://blog.csdn.net/zouchengli/article/details/107783675

版权

shell 同时被 3 个专栏收录

11 篇文章 0 订阅

订阅专栏

k8s

10 篇文章 0 订阅

订阅专栏

keepalived

2 篇文章 0 订阅

订阅专栏

前言

之前我有写过普通集群搭建，没有给master做高可用，于是很多给我评论如何弄ha。今天我们就开始怎么去搭建一个高可用的Kuberneters 集群吧！

准备机器或虚拟机

我这里使用六台CentOS虚拟机，你们机器少的话可以删减两台worker，我这里主要还是讲解master的高可用。

镜像地址：http://mirrors.aliyun.com/centos/7/isos/x86_64/CentOS-7-x86_64-Minimal-2003.iso

IP	HOSTNAME	CPU	MEMORY	DISK
172.16.0.77	master00	>=2	>=4G	>=8G
172.16.0.78	master01	>=2	>=4G	>=8G
172.16.0.79	master02	>=2	>=4G	>=8G
172.16.0.80	worker00	>=2	>=4G	>=16G
172.16.0.81	worker01	>=2	>=4G	>=16G
172.16.0.82	worker02	>=2	>=4G	>=16G

详细安装步骤与上一篇大同小异

我这里不多做太多介绍了，直接上脚本。需要注意的是我这里的机器与外网隔离的，你们自己ip不一定要按照我的来，自己设置成桥接有网络的配置。

每台机器或虚拟机都执行如下脚本：

systemctl disable postfix --now && systemctl disable firewalld --now
setenforce 0 && sed -i "s/^SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config
swapoff -a && sed -i 's/.*swap.*/#&/' /etc/fstab
cat > /etc/sysctl.d/k8s.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system
cd /etc/yum.repos.d/
mkdir backup
mv ./*.repo backup
curl -O http://mirrors.aliyun.com/repo/Centos-7.repo
curl -O https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
EOF
yum clean all && yum makecache
yum -y update
yum -y install docker-ce docker-ce-cli containerd.io kubelet kubeadm kubectl
systemctl enable docker --now
cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ]
}
EOF
mkdir -p /etc/systemd/system/docker.service.d
systemctl daemon-reload && systemctl restart docker
systemctl enable kubelet --now

在各master节点执行如下脚本：

yum -y install keepalived

在master00节点编辑kubeadm-config.yaml文件并保存：

apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v1.18.6
imageRepository: registry.cn-shanghai.aliyuncs.com/k8sgcrio_containers
apiServer:
  certSANs:
  - master00
  - master01
  - master02
  - vip
  - 172.16.0.77
  - 172.16.0.78
  - 172.16.0.79
  - 172.16.0.76
controlPlaneEndpoint: "172.16.0.76:6443"
#etcd:
    #external:
        #endpoints:
        #- http://172.16.0.160:2379
        #- http://172.16.0.161:2379
        #- http://172.16.0.162:2379
networking:
  dnsDomain: cluster.local
  podSubnet: 10.244.0.0/16
  serviceSubnet: 10.96.0.0/12

其中 apiServer certSANS 填写master host名和ip以及虚拟ip(VIP)
controlPlaneEndpoint: 可以填VIP，也可以填master其中一个ip。其实个人推荐在准备两台机器搭建nginx一主一备，直接负载均衡转发到三台master即可。
etcd默认使用本地，也可以使用外部etcd集群,搭建方式可以参考我我的Etcd集群搭建：。我这里使用堆叠etcd方式。

在master00节点执行如下脚本：

mv /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.bak
cat << EOF > /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id master00
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 50
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.16.0.76
    }
}

EOF
systemctl enable keepalived --now
kubeadm init --config=kubeadm-config.yaml
输出结果(一定要记得保存哦，后面会用到！！！)：
此处忽略前半部分
Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of control-plane nodes by copying certificate authorities
and service account keys on each node and then running the following as root:

  kubeadm join 172.16.0.76:6443 --token ymrykq.ra79tq8u29ovhjky \
    --discovery-token-ca-cert-hash sha256:a0f42468b84d3826e92de594416652a5d9b2f757923b1a8ac2a0d7accc80ca75 \
    --control-plane 

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 172.16.0.76:6443 --token ymrykq.ra79tq8u29ovhjky \
    --discovery-token-ca-cert-hash sha256:a0f42468b84d3826e92de594416652a5d9b2f757923b1a8ac2a0d7accc80ca75

上面其实输出结果都告诉你了，我这里再贴一遍：

mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

此时我们使用客户端获取节点信息：

[root@master00 ~]# kubectl get nodes
NAME       STATUS     ROLES    AGE     VERSION
master00   NotReady   master   5m24s   v1.18.6

上面我们看到状态为未就绪，原因是还有一步网络插件没部署，现在我们部署网络插件：

这里的kube-flannel.yml文件请看我上篇文章。

[root@master00 ~]# kubectl apply -f kube-flannel.yml
podsecuritypolicy.policy/psp.flannel.unprivileged created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds-amd64 created
daemonset.apps/kube-flannel-ds-arm64 created
daemonset.apps/kube-flannel-ds-arm created
daemonset.apps/kube-flannel-ds-ppc64le created
daemonset.apps/kube-flannel-ds-s390x created

免密登录设置在master00上操作：

ssh-keygen
ssh-copy-id master01
ssh-copy-id master02

复制证书相关文件到其他master节点：

ssh master01 'mkdir -p /etc/kubernetes/pki/etcd/'
ssh master02 'mkdir -p /etc/kubernetes/pki/etcd/'
cd /etc/kubernetes/pki/
scp -r ca.* sa.* front-proxy-ca.* master01:$PWD
scp -r ca.* sa.* front-proxy-ca.* master02:$PWD
cd /etc/kubernetes/pki/etcd/
scp -r ca.* master01:$PWD
scp -r ca.* master02:$PWD

在master01节点执行如下脚本：

mv /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.bak
cat << EOF > /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id master01
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 50
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.16.0.76
    }
}
EOF
systemctl enable keepalived --now
kubeadm join 172.16.0.76:6443 --token ymrykq.ra79tq8u29ovhjky \
    --discovery-token-ca-cert-hash sha256:a0f42468b84d3826e92de594416652a5d9b2f757923b1a8ac2a0d7accc80ca75 \
    --control-plane
输出结果：
忽略前半部分
This node has joined the cluster and a new control plane instance was created:

* Certificate signing request was sent to apiserver and approval was received.
* The Kubelet was informed of the new secure connection details.
* Control plane (master) label and taint were applied to the new node.
* The Kubernetes control plane instances scaled up.
* A new etcd member was added to the local/stacked etcd cluster.

To start administering your cluster from this node, you need to run the following as a regular user:

	mkdir -p $HOME/.kube
	sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
	sudo chown $(id -u):$(id -g) $HOME/.kube/config

Run 'kubectl get nodes' to see this node join the cluster.

在master02节点执行如下脚本：

mv /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.bak
cat << EOF > /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id master02
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 50
    priority 80
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.16.0.76
    }
}
EOF
systemctl enable keepalived --now
kubeadm join 172.16.0.76:6443 --token ymrykq.ra79tq8u29ovhjky \
    --discovery-token-ca-cert-hash sha256:a0f42468b84d3826e92de594416652a5d9b2f757923b1a8ac2a0d7accc80ca75 \
    --control-plane
输出结果：
忽略前半部分
This node has joined the cluster and a new control plane instance was created:

* Certificate signing request was sent to apiserver and approval was received.
* The Kubelet was informed of the new secure connection details.
* Control plane (master) label and taint were applied to the new node.
* The Kubernetes control plane instances scaled up.
* A new etcd member was added to the local/stacked etcd cluster.

To start administering your cluster from this node, you need to run the following as a regular user:

	mkdir -p $HOME/.kube
	sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
	sudo chown $(id -u):$(id -g) $HOME/.kube/config

Run 'kubectl get nodes' to see this node join the cluster.

在master00上检查集群情况：

[root@master00 ~]# kubectl get nodes
NAME      STATUS   ROLES    AGE     VERSION
master00  Ready    master   37m     v1.18.6
master01  Ready    master   10m     v1.18.6
master02  Ready    master   9m53s   v1.18.6

大家可以看到master都成功加入集群了，这里是三个master，当master00挂掉的时候，keepalived的lvs技术会根据优先级虚拟vip。也就是我们前面配置到的172.16.0.76！当master00恢复的时候lvs会把虚拟vip交给master00。后面worker加入就更加简单了，直接复制上面加入命令。

在各个worker节点执行如下命令：

[root@worker00 ~]# kubeadm join 172.16.0.76:6443 --token ymrykq.ra79tq8u29ovhjky \
>     --discovery-token-ca-cert-hash sha256:a0f42468b84d3826e92de594416652a5d9b2f757923b1a8ac2a0d7accc80ca75 
W0804 16:19:40.260197    1645 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
[preflight] Running pre-flight checks
	[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.18" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

在返回到master00执行如下命令：

[root@master00 ~]# kubectl get nodes
NAME      STATUS   ROLES    AGE   VERSION
master00  Ready    master   77m   v1.18.6
master01  Ready    master   50m   v1.18.6
master02  Ready    master   50m   v1.18.6
worker00  Ready    <none>   42s   v1.18.6
worker01  Ready    <none>   19s   v1.18.6
worker02  Ready    <none>   10s   v1.18.6

好了，高可用集群搭建完毕，有什么问题继续下面评论。

chengli.zou

关注

1
点赞
踩
0

收藏

觉得还不错? 一键收藏
4
评论
kubernetes v1.18.6 高可用集群集群安装说明

前言之前我有写过普通集群搭建，没有给master做高可用，于是很多给我评论如何弄ha。今天我们就开始怎么去搭建一个高可用的Kuberneters 集群吧！准备机器或虚拟机我这里使用六台CentOS虚拟机，你们机器少的话可以删减两台worker，我这里主要还是讲解master的高可用。镜像地址：http://mirrors.aliyun.com/centos/7/isos/x86_64/CentOS-7-x86_64-Minimal-2003.isoIP HOSTNAME
复制链接

扫一扫