Kubernetes集群部署

一、环境准备

1.服务器规划

master 192.168.18.201 2c4G Centos 7.6

node1 192.168.18.204 1c4G Centos 7.6

node2 192.168.18.205 1c4G Centos 7.6

node3 192.168.18.206 1c4G Centos 7.6

2.软件准备

Docker 19.03.15

Kubelet 1.22.3

Kubeadm

Kubectl

3.编辑hosts文件

echo "192.168.18.201  master" >> /etc/hosts
echo "192.168.18.204  node1" >> /etc/hosts
echo "192.168.18.205  node2" >> /etc/hosts
echo "192.168.18.206  node3" >> /etc/hosts

4.关闭防火墙和Selinux

systemctl  stop firewalld  && systemctl  disable firewalld
setenforce 0
sed -i '/^SELINUX=/s/enforcing/disabled/' /etc/selinux/config

5.关闭SWAP

swapoff -a 
sed  -ri  's@(.*swap.*)@#\1@g' /etc/fstab

6.内核修改

开启内核IPV4转发

modprobe br_netfilter

配置内核参数

cat >/etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables=1
net.bridge.bridge-nf-call-iptables=1
net.ipv4.ip_forward=1
vm.swappiness=0
EOF

sysctl  -p /etc/sysctl.d/k8s.conf

7.安装IPVS

yum install ipset  ipvsadm -y 

加载ipvs模块

cat >/etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
EOF

修改权限

chmod 775 /etc/sysconfig/modules/ipvs.modules 
bash /etc/sysconfig/modules/ipvs.modules
lsmod | grep -e ip_vs -e nf_conntrack_ipv4

8.时钟同步

安装chrony

yum install -y chrony 

启用chrony

systemctl enable  chronyd --now

同步时间

chronyc sources 

二、安装集群组件

1.安装Docker

清理旧docker

yum remove docker* -y 

安装依赖

yum install  -y yum-utils
yum install -y  https://mirrors.aliyun.com/centos/7.9.2009/extras/x86_64/Packages/container-selinux-2.119.1-1.c57a6f9.el7.noarch.rpm

配置阿里云的docker yum源

yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

安装Docker 并配置镜像加速

yum install docker-ce-19.03.15 -y 
cat > /etc/docker/daemon.json << EOF
{
"registry-mirrors": ["https://gqs7xcfd.mirror.aliyuncs.com","https://hub-mirror.c.163.com"],
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
},
"storage-driver": "overlay2"
}
EOF

启动Docker

systemctl  daemon-reload 
systemctl  enable docker --now

2.安装k8s集群软件

配置k8s 阿里yum源

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enable=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
      http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

添加contos源

 wget -O /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo

安装k8s

yum install -y kubelet-1.22.3 kubeadm-1.22.3 kubectl-1.22.3

启动kubelet 并加入开机自启

kubelet version 

systemctl enable kubelet --now

三、集群初始化

1.下载docker镜像

查看初始化所需镜像

[root@master yum.repos.d]# kubeadm config images list --kubernetes-version v1.22.3
k8s.gcr.io/kube-apiserver:v1.22.3
k8s.gcr.io/kube-controller-manager:v1.22.3
k8s.gcr.io/kube-scheduler:v1.22.3
k8s.gcr.io/kube-proxy:v1.22.3
k8s.gcr.io/pause:3.5
k8s.gcr.io/etcd:3.5.0-0
k8s.gcr.io/coredns/coredns:v1.8.4

重定向镜像位置

 kubeadm config images list --image-repository registry.aliyuncs.com/google_containers --kubernetes-version 1.22.3 
registry.aliyuncs.com/google_containers/kube-apiserver:v1.22.3
registry.aliyuncs.com/google_containers/kube-controller-manager:v1.22.3
registry.aliyuncs.com/google_containers/kube-scheduler:v1.22.3
registry.aliyuncs.com/google_containers/kube-proxy:v1.22.3
registry.aliyuncs.com/google_containers/pause:3.5
registry.aliyuncs.com/google_containers/etcd:3.5.0-0
registry.aliyuncs.com/google_containers/coredns:v1.8.4

 kubeadm config images list --image-repository oldxu3957 --kubernetes-version v1.22.3
oldxu3957/kube-apiserver:v1.22.3
oldxu3957/kube-controller-manager:v1.22.3
oldxu3957/kube-scheduler:v1.22.3
oldxu3957/kube-proxy:v1.22.3
oldxu3957/pause:3.5
oldxu3957/etcd:3.5.0-0
oldxu3957/coredns:v1.8.4

拉取镜像

[root@master yum.repos.d]# kubeadm config images pull --image-repository registry.aliyuncs.com/google_containers --kubernetes-version 1.22.3 
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-apiserver:v1.22.3
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-controller-manager:v1.22.3
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-scheduler:v1.22.3
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-proxy:v1.22.3
[config/images] Pulled registry.aliyuncs.com/google_containers/pause:3.5
[config/images] Pulled registry.aliyuncs.com/google_containers/etcd:3.5.0-0
[config/images] Pulled registry.aliyuncs.com/google_containers/coredns:v1.8.4

查看镜像

[root@master ~]# docker image list
REPOSITORY                                                        TAG                 IMAGE ID            CREATED             SIZE
registry.aliyuncs.com/google_containers/kube-apiserver            v1.22.3             53224b502ea4        2 years ago         128MB
registry.aliyuncs.com/google_containers/kube-controller-manager   v1.22.3             05c905cef780        2 years ago         122MB
registry.aliyuncs.com/google_containers/kube-scheduler            v1.22.3             0aa9c7e31d30        2 years ago         52.7MB
registry.aliyuncs.com/google_containers/kube-proxy                v1.22.3             6120bd723dce        2 years ago         104MB
registry.aliyuncs.com/google_containers/etcd                      3.5.0-0             004811815584        3 years ago         295MB
registry.aliyuncs.com/google_containers/coredns                   v1.8.4              8d147537fb7d        3 years ago         47.6MB
registry.aliyuncs.com/google_containers/pause                     3.5                 ed210e3e4a5b        3 years ago         683kB

2.初始化master节点

kubeadm init \
  --apiserver-advertise-address=192.168.18.201 \
  --image-repository registry.aliyuncs.com/google_containers\
  --kubernetes-version 1.22.3 \
  --service-cidr=10.96.0.0/16 \
  --pod-network-cidr=10.10.0.0/16
  

配置文件

[root@master docker]#  mkdir -p $HOME/.kube
[root@master docker]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@master docker]#  sudo chown $(id -u):$(id -g) $HOME/.kube/config

3.初始化nodes节点

三个节点加入集群

kubeadm join 192.168.18.201:6443 --token cf5nld.e9u0xy880syurb2f \
        --discovery-token-ca-cert-hash sha256:e3885495751c131d3d8740c833bc0ebc3313ac10fea242df4150bee50b04b85f 

4.查看节点状态

[root@master ~]# kubectl get nodes
NAME     STATUS     ROLES                  AGE     VERSION
master   NotReady   control-plane,master   9m38s   v1.22.3
node1    NotReady   <none>                 113s    v1.22.3
node2    NotReady   <none>                 98s     v1.22.3
node3    NotReady   <none>                 85s     v1.22.3

5.安装Flannel 网络插件

下载插件

wget https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml

修改POD网段地址

sed -i 's#10.244.0.0/16#10.10.0.0/16#g' kube-flannel.yml

应用flannel

[root@master .kube]# kubectl  apply -f kube-flannel.yml 
namespace/kube-flannel created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created

查看pod状态

[root@master .kube]# kubectl get pods -n kube-flannel
NAME                    READY   STATUS    RESTARTS   AGE
kube-flannel-ds-28xvp   1/1     Running   0          18m
kube-flannel-ds-86562   1/1     Running   0          10m
kube-flannel-ds-d8l9n   1/1     Running   0          18m
kube-flannel-ds-zlntv   1/1     Running   0          18m

查看节点状态

[root@master .kube]# kubectl get nodes
NAME     STATUS   ROLES                  AGE   VERSION
master   Ready    control-plane,master   20m   v1.22.3
node1    Ready    <none>                 19m   v1.22.3
node2    Ready    <none>                 19m   v1.22.3
node3    Ready    <none>                 18m   v1.22.3

6.集群命令自动补全

 yum install  bash-completion -y
 echo 'source <(kubectl completion bash)'  >>~/.bashrc

四、集群状态监测

1.查看运行的POD

[root@master .kube]# kubectl get pods -A
NAMESPACE      NAME                             READY   STATUS    RESTARTS      AGE
kube-flannel   kube-flannel-ds-28xvp            1/1     Running   0             32m
kube-flannel   kube-flannel-ds-86562            1/1     Running   0             24m
kube-flannel   kube-flannel-ds-d8l9n            1/1     Running   0             32m
kube-flannel   kube-flannel-ds-zlntv            1/1     Running   0             32m
kube-system    coredns-7f6cbbb7b8-pbrzj         1/1     Running   0             35m
kube-system    coredns-7f6cbbb7b8-zsfbj         1/1     Running   0             35m
kube-system    etcd-master                      1/1     Running   6             35m
kube-system    kube-apiserver-master            1/1     Running   6             35m
kube-system    kube-controller-manager-master   1/1     Running   5             35m
kube-system    kube-proxy-299lb                 1/1     Running   0             35m
kube-system    kube-proxy-69kxm                 1/1     Running   1 (25m ago)   34m
kube-system    kube-proxy-rgmwk                 1/1     Running   1 (26m ago)   34m
kube-system    kube-proxy-tqsx4                 1/1     Running   1 (27m ago)   34m
kube-system    kube-scheduler-master            1/1     Running   4             35m
[root@master .kube]# 

2.创建Nginx 测试集群

kubectl create  deployment nginx --image=nginx:latest --replicas=3