Centos7 搭建k8s单master集群（版本1.20.1-0）

Centos7 搭建k8s单master集群（版本1.20.1-0）

机器和环境

系统：Centos7.4
机器：

192.168.131.130    k8s-master
192.168.131.141    k8s-node1
192.168.131.142    k8s-node2

机器性能：
k8s-master： 2核 4G
k8s-node1： 4核 8G
k8s-node2： 4核 8G
注意： cpu最低要2核，内存最低要2G，自己看机器配置用虚拟机划分资源

注意

以下操作不指明操作那台主机，就是所有机器都要执行一边

准备机器环境

修改服务器名

k8s-master机器执行：hostnamectl set-hostname k8s-master
k8s-node1机器执行：hostnamectl set-hostname k8s-node1
k8s-node2机器执行：hostnamectl set-hostname k8s-node2
查询结果：hostname

添加路径映射

• vi /etc/hosts 向里面添加

  192.168.131.130    k8s-master
  192.168.131.141    k8s-node1
  192.168.131.142    k8s-node2
  

注意：这个ip要根据自己电脑情况

• 检查：
  - ping k8s-master
  - ping k8s-node1
  - ping k8s-node2

关闭防火墙

systemctl stop firewalld && systemctl disable firewalld

关闭selinux

1. 临时关闭
   setenforce 0
2. 永久关闭
   修改配置/etc/selinux/config,将SELINUX设置为disabled
   vi /etc/selinux/config
   

关闭swap

1. 临时关闭
   swapoff -a

2. 永久关闭

   • vi /etc/fstab
   • 修改/etc/fstab，注释最后一行
   • 

3. 检查是否关闭
   swap都是零则是swap关闭
   

开启 bridge-nf

修改vi /etc/sysctl.conf，末尾添加如下配置：

net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-arptables = 1

重启验证

• 重启服务器：reboot now
• 防火墙是否关闭： systemctl status firewalld
• selinux是否关闭： getenforce
• swap是否关闭： free -m

安装docker

• Docker安装博客
• 简化命令:

yum install docker -y && systemctl start docker && systemctl enable docker

时间同步

• 安装时间同步：yum install -y ntp ntpdate
• 开启时间同步： ntpdate cn.pool.ntp.org
• 开启设置开机启动： systemctl start ntpd && systemctl enable ntpd

安装Kubernetes基本组件

设置Kubernetes国内源

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

安装相关组件

• 安装kubelet kubeadm kubectl

yum install -y kubelet-1.20.1-0 kubeadm-1.20.1-0 kubectl-1.20.1-0 --disableexcludes=kubernetes

• 设置开机启动kubelet
  systemctl start kubelet && systemctl enable kubelet

安装Master节点

注意：该部操作在k8s-master执行

创建集群

• 执行脚本

  kubeadm init \
    --kubernetes-version=v1.20.1 \
    --pod-network-cidr=10.244.0.0/16 \
    --image-repository registry.aliyuncs.com/google_containers \
    --apiserver-advertise-address 192.168.131.130 \
    --v=6
  

• 参数说明：

    kubernetes-version：要安装的版本
    pod-network-cidr：负载容器的子网网段
    image-repository：指定镜像仓库（由于从阿里云拉镜像，解决了k8s.gcr.io镜像拉不下来的问题）
    apiserver-advertise-address：节点绑定的服务器ip（多网卡可以用这个参数指定ip）
    v=6：这个参数我还没具体查文档，用法是初始化过程显示详细内容，部署过程如果有问题看详细信息很多时候能找到问题所在
  

• 有可能存在镜像拉不下来，这里可以查看需要镜像版本，可以考虑手动去拉取镜像，然后tag镜像
  kubeadm config images list

• 报错解决：

  • 根据报错日志，解决报错
  • 重置取消init: kubeadm reset
  • 再次执行init

• 安装成功后
  

• 添加到环境变量里面

echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile
source ~/.bash_profile

• 检查状态： kubectl get cs

    发现scheduler和controller-manager的状态为Unhealthy，一般
    是因为kube-scheduler和kube-controller-manager组件配置默认
    禁用了非安全端口，修改配置文件中的port=0配置，配置文件路
    径：
    vi /etc/kubernetes/manifests/kube-scheduler.yaml
    vi /etc/kubernetes/manifests/kube-controller-manager.yaml
  

  
  

• 重启kubelet：systemctl restart kubelet

• 再次检查状态： kubectl get cs
  

安装Flannel网络插件

1. 创建目录：mkdir -p /opt/yaml

2. 创建yaml文件

   cd  /opt/yaml && touch kube-flannel.yaml && vi kube-flannel.yaml
   

3. 讲下面内容粘贴到文件里面

   ---
   apiVersion: policy/v1beta1
   kind: PodSecurityPolicy
   metadata:
     name: psp.flannel.unprivileged
     annotations:
       seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default
       seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default
       apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
       apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default
   spec:
     privileged: false
     volumes:
     - configMap
     - secret
     - emptyDir
     - hostPath
     allowedHostPaths:
     - pathPrefix: "/etc/cni/net.d"
     - pathPrefix: "/etc/kube-flannel"
     - pathPrefix: "/run/flannel"
     readOnlyRootFilesystem: false
     # Users and groups
     runAsUser:
       rule: RunAsAny
     supplementalGroups:
       rule: RunAsAny
     fsGroup:
       rule: RunAsAny
     # Privilege Escalation
     allowPrivilegeEscalation: false
     defaultAllowPrivilegeEscalation: false
     # Capabilities
     allowedCapabilities: ['NET_ADMIN', 'NET_RAW']
     defaultAddCapabilities: []
     requiredDropCapabilities: []
     # Host namespaces
     hostPID: false
     hostIPC: false
     hostNetwork: true
     hostPorts:
     - min: 0
       max: 65535
     # SELinux
     seLinux:
       # SELinux is unused in CaaSP
       rule: 'RunAsAny'
   ---
   kind: ClusterRole
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
     name: flannel
   rules:
   - apiGroups: ['extensions']
     resources: ['podsecuritypolicies']
     verbs: ['use']
     resourceNames: ['psp.flannel.unprivileged']
   - apiGroups:
     - ""
     resources:
     - pods
     verbs:
     - get
   - apiGroups:
     - ""
     resources:
     - nodes
     verbs:
     - list
     - watch
   - apiGroups:
     - ""
     resources:
     - nodes/status
     verbs:
     - patch
   ---
   kind: ClusterRoleBinding
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
     name: flannel
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: ClusterRole
     name: flannel
   subjects:
   - kind: ServiceAccount
     name: flannel
     namespace: kube-system
   ---
   apiVersion: v1
   kind: ServiceAccount
   metadata:
     name: flannel
     namespace: kube-system
   ---
   kind: ConfigMap
   apiVersion: v1
   metadata:
     name: kube-flannel-cfg
     namespace: kube-system
     labels:
       tier: node
       app: flannel
   data:
     cni-conf.json: |
       {
         "name": "cbr0",
         "cniVersion": "0.3.1",
         "plugins": [
           {
             "type": "flannel",
             "delegate": {
               "hairpinMode": true,
               "isDefaultGateway": true
             }
           },
           {
             "type": "portmap",
             "capabilities": {
               "portMappings": true
             }
           }
         ]
       }
     net-conf.json: |
       {
         "Network": "10.244.0.0/16",
         "Backend": {
           "Type": "vxlan"
         }
       }
   ---
   apiVersion: apps/v1
   kind: DaemonSet
   metadata:
     name: kube-flannel-ds
     namespace: kube-system
     labels:
       tier: node
       app: flannel
   spec:
     selector:
       matchLabels:
         app: flannel
     template:
       metadata:
         labels:
           tier: node
           app: flannel
       spec:
         affinity:
           nodeAffinity:
             requiredDuringSchedulingIgnoredDuringExecution:
               nodeSelectorTerms:
               - matchExpressions:
                 - key: kubernetes.io/os
                   operator: In
                   values:
                   - linux
         hostNetwork: true
         priorityClassName: system-node-critical
         tolerations:
         - operator: Exists
           effect: NoSchedule
         serviceAccountName: flannel
         initContainers:
         - name: install-cni
           image: quay.io/coreos/flannel:v0.13.1-rc1
           command:
           - cp
           args:
           - -f
           - /etc/kube-flannel/cni-conf.json
           - /etc/cni/net.d/10-flannel.conflist
           volumeMounts:
           - name: cni
             mountPath: /etc/cni/net.d
           - name: flannel-cfg
             mountPath: /etc/kube-flannel/
         containers:
         - name: kube-flannel
           image: quay.io/coreos/flannel:v0.13.1-rc1
           command:
           - /opt/bin/flanneld
           args:
           - --ip-masq
           - --kube-subnet-mgr
           resources:
             requests:
               cpu: "100m"
               memory: "50Mi"
             limits:
               cpu: "100m"
               memory: "50Mi"
           securityContext:
             privileged: false
             capabilities:
               add: ["NET_ADMIN", "NET_RAW"]
           env:
           - name: POD_NAME
             valueFrom:
               fieldRef:
                 fieldPath: metadata.name
           - name: POD_NAMESPACE
             valueFrom:
               fieldRef:
                 fieldPath: metadata.namespace
           volumeMounts:
           - name: run
             mountPath: /run/flannel
           - name: flannel-cfg
             mountPath: /etc/kube-flannel/
         volumes:
         - name: run
           hostPath:
             path: /run/flannel
         - name: cni
           hostPath:
             path: /etc/cni/net.d
         - name: flannel-cfg
           configMap:
             name: kube-flannel-cfg
   

4. 将quay.io换成quay.mirrors.ustc.edu.cn（中科大）的镜像

   sed -i 's#quay.io/coreos/flannel#quay.mirrors.ustc.edu.cn/coreos/flannel#' /opt/yaml/kube-flannel.yaml
   

5. 执行安装命令

   kubectl apply -f /opt/yaml/kube-flannel.yaml
   

6. 查看节点状态：kubectl get nodes
   

安装Node

注意： 该步操作只在node节点执行

环境变量配置

1. 配置文件从master拷贝到node,注意：在master节点执行

   scp /etc/kubernetes/admin.conf root@k8s-node1://etc/kubernetes/
   scp /etc/kubernetes/admin.conf root@k8s-node2://etc/kubernetes/
   

2. 给node节点添加环境，注意： 在node节点执行

   echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile
   source ~/.bash_profile
   

Node节点加入Master

1. 因为修改过一些配置，重启过master，这里为了防止报错，我们重新生成加入token，在master机器执行:
   kubeadm token create --print-join-command
   

2. 在node节点执行：

   kubeadm join 192.168.131.130:6443 --token d59oyt.zleslhkqdbtiq5tg     --discovery-token-ca-cert-hash sha256:28cb37c295368b14070590b85703fadea268edef7893ee04649f7d5e4e9124ab --v=6
   

3. 注意结尾有个 --v=6 要自己手动加上去

4. 在master执行： kubectl get nodes
   

报错处理

节点状态NotReady排查

1. 可能是某个docker镜像没起来
   • 执行： kubectl get pods -o wide --all-namespaces
     
     • 找到0/1类型： docker logs 容器id， 看看为什么起不来