K8S单机部署-01.单机部署K8S

转载请注明出处

相关链接

• K8S单机部署-00.旧版本卸载
• K8S单机部署-01.单机部署K8S
• K8S单机部署-02.Kuboard面板部署
• K8S单机部署-03.验证K8S的第一个Deployment
• K8S单机部署-04.NFS服务器搭建
• K8S单机部署-05.静态持久化之PV、PVC(基于NFS)
• K8S单机部署-06.动态持久化StorageClass(基于NFS)
• K8S单机部署-07.调整nodePort端口范围
• K8S单机部署-08.安装ingress-nginx-controller
• K8S单机部署-10.增加自定义域名解析
• K8S单机部署-11.安装Kubernetes Metrics Server监控

如果之前部署过, 需要先彻底卸载
如果之前部署过, 需要先彻底卸载
如果之前部署过, 需要先彻底卸载

0.部署环境

Centos 7.9

1.查看系统版本信息以及修改配置信息

1.1 查看cpu信息 k8s安装至少需要2核2G的环境，否则会安装失败

lscpu

1.2 安装k8s时，临时关闭swap ，如果不关闭在执行kubeadm部分命令会报错

swapoff -a

1.3 安装k8s时，可以临时关闭selinux，减少额外配置

setenforce 0

1.4 关闭防火墙

systemctl stop firewalld
systemctl disable firewalld

1.5 设置网桥参数

cat << EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF

1.6 修改hosts文件 方便查看域名映射

vim /etc/hosts

 追加以下内容, 将IP更换为真实IP
192.168.21.223 master

让修改生效

service network restart
或者
/etc/init.d/network restart

1.7 修改hostname

hostnamectl set-hostname master

2.安装docker

2.1 使用阿里云安装

yum install -y yum-utils device-mapper-persistent-data lvm2

设置repo源为阿里云

yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum -y install docker-ce

2.2 修改docker的 /etc/docker/daemon.json文件
如果没有该文件则直接创建

sudo mkdir -p /etc/docker

设置国内阿里云docker源，地址改为自己在阿里云容器镜像服务申请的地址即可。

设置cgroupdriver为systemd，这步尤为重要(会导致后面初始化kubectl失败)，笔者就是因为没设置走了很多弯路

sudo tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["https://zydpncxr.mirror.aliyuncs.com"],
  "exec-opts": ["native.cgroupdriver=systemd"]
}
EOF

2.3 修改完成后 重启docker ，使docker与kubelet的cgroup 驱动一致

sudo systemctl daemon-reload

sudo systemctl restart docker

2.4 设置docker开机启动

systemctl enable docker

3.安装kubeadm kubelet kubectl

3.1 配置k8s下载资源配置文件（阿里云yum源）

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

3.2 安装 kubelet kubeadm kubectl

yum install -y kubelet-1.23.5 kubeadm-1.23.5 kubectl-1.23.5

• kubelet ：运行在cluster，负责启动pod管理容器
• kubeadm ：k8s快速构建工具，用于初始化cluster
• kubectl ：k8s命令工具，部署和管理应用，维护组件

3.2.1 查看是否安装成功

kubelet --version
kubectl version
kubeadm version

3.3 启动kubelet

systemctl daemon-reload

systemctl start kubelet

设置kubelet开机启动

systemctl enable kubelet

3.4 拉取init-config配置 并修改配置
init-config 主要由 api server、etcd、scheduler、controller-manager、coredns等镜像构成

kubeadm config print init-defaults > init-config.yaml

3.4.1 修改 刚才拉取的init-config.yaml文件，共3处修改

• localAPIEndpoint.advertiseAddress
• nodeRegistration.name
• imageRepository

apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 192.168.21.223   //修改1 master节点IP地址 可cat /etc/hosts 看到
  bindPort: 6443
nodeRegistration:
  criSocket: /var/run/dockershim.sock
  imagePullPolicy: IfNotPresent
  name: master                       //修改2 master节点node的名称
  taints: null
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns: {}
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers       //修改3 修改为阿里云地址
kind: ClusterConfiguration
kubernetesVersion: 1.23.0
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.96.0.0/12
scheduler: {}

3.5 拉取k8s相关镜像

kubeadm config images pull --config=init-config.yaml

如果镜像拉取失败，可以通过命令列出需要的镜像后逐个拉取，命令如下：

kubeadm config images list --config init-config.yaml

显示如下（版本可能有所差别）：

[root@master install]# kubeadm config images pull --config=init-config.yaml
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-apiserver:v1.23.0
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-controller-manager:v1.23.0
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-scheduler:v1.23.0
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-proxy:v1.23.0
[config/images] Pulled registry.aliyuncs.com/google_containers/pause:3.6
[config/images] Pulled registry.aliyuncs.com/google_containers/etcd:3.5.1-0
[config/images] Pulled registry.aliyuncs.com/google_containers/coredns:v1.8.6

3.6 运行kubeadm init安装master节点
把192.168.21.223替换成你自己的网卡IP

kubeadm init --apiserver-advertise-address=192.168.21.223 --apiserver-bind-port=6443 --pod-network-cidr=10.244.0.0/16  --service-cidr=10.96.0.0/12 --kubernetes-version=1.23.5 --image-repository registry.aliyuncs.com/google_containers

不出意外的话，到此你已经看到安装成功的信息了, 基本长下面这样

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.21.223:6443 --token k0n5rz.9ojc844imvkz4wjn \
        --discovery-token-ca-cert-hash sha256:2aff82ae621fd95c1fbd2a9e6d814139da98618832bbac22cc97c156ad64fab7

根据上面的提示执行下面的命令

 mkdir -p $HOME/.kube
 sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
 sudo chown $(id -u):$(id -g) $HOME/.kube/config

3.7 如果安装失败可以执行命令重新安装

kubeadm reset # 重置集群
# kubeadm reset -f 
# 然后再重新执行上面提到的kubeadm init命令

4.部署网络插件kube-flannel

下载部署文件

wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

安装插件

kubectl apply -f kube-flannel.yml

如果网络不好下载不下来也可以直接复制以下内容新建yml文件执行

---
kind: Namespace
apiVersion: v1
metadata:
  name: kube-flannel
  labels:
    pod-security.kubernetes.io/enforce: privileged
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: flannel
rules:
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - nodes/status
  verbs:
  - patch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: flannel
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: flannel
subjects:
- kind: ServiceAccount
  name: flannel
  namespace: kube-flannel
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: flannel
  namespace: kube-flannel
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: kube-flannel-cfg
  namespace: kube-flannel
  labels:
    tier: node
    app: flannel
data:
  cni-conf.json: |
    {
      "name": "cbr0",
      "cniVersion": "0.3.1",
      "plugins": [
        {
          "type": "flannel",
          "delegate": {
            "hairpinMode": true,
            "isDefaultGateway": true
          }
        },
        {
          "type": "portmap",
          "capabilities": {
            "portMappings": true
          }
        }
      ]
    }
  net-conf.json: |
    {
      "Network": "10.244.0.0/16",
      "Backend": {
        "Type": "vxlan"
      }
    }
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: kube-flannel-ds
  namespace: kube-flannel
  labels:
    tier: node
    app: flannel
spec:
  selector:
    matchLabels:
      app: flannel
  template:
    metadata:
      labels:
        tier: node
        app: flannel
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/os
                operator: In
                values:
                - linux
      hostNetwork: true
      priorityClassName: system-node-critical
      tolerations:
      - operator: Exists
        effect: NoSchedule
      serviceAccountName: flannel
      initContainers:
      - name: install-cni-plugin
       #image: flannelcni/flannel-cni-plugin:v1.1.0 for ppc64le and mips64le (dockerhub limitations may apply)
        image: docker.io/rancher/mirrored-flannelcni-flannel-cni-plugin:v1.1.0
        command:
        - cp
        args:
        - -f
        - /flannel
        - /opt/cni/bin/flannel
        volumeMounts:
        - name: cni-plugin
          mountPath: /opt/cni/bin
      - name: install-cni
       #image: flannelcni/flannel:v0.20.0 for ppc64le and mips64le (dockerhub limitations may apply)
        image: docker.io/rancher/mirrored-flannelcni-flannel:v0.20.0
        command:
        - cp
        args:
        - -f
        - /etc/kube-flannel/cni-conf.json
        - /etc/cni/net.d/10-flannel.conflist
        volumeMounts:
        - name: cni
          mountPath: /etc/cni/net.d
        - name: flannel-cfg
          mountPath: /etc/kube-flannel/
      containers:
      - name: kube-flannel
       #image: flannelcni/flannel:v0.20.0 for ppc64le and mips64le (dockerhub limitations may apply)
        image: docker.io/rancher/mirrored-flannelcni-flannel:v0.20.0
        command:
        - /opt/bin/flanneld
        args:
        - --ip-masq
        - --kube-subnet-mgr
        resources:
          requests:
            cpu: "100m"
            memory: "50Mi"
          limits:
            cpu: "100m"
            memory: "50Mi"
        securityContext:
          privileged: false
          capabilities:
            add: ["NET_ADMIN", "NET_RAW"]
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: EVENT_QUEUE_DEPTH
          value: "5000"
        volumeMounts:
        - name: run
          mountPath: /run/flannel
        - name: flannel-cfg
          mountPath: /etc/kube-flannel/
        - name: xtables-lock
          mountPath: /run/xtables.lock
      volumes:
      - name: run
        hostPath:
          path: /run/flannel
      - name: cni-plugin
        hostPath:
          path: /opt/cni/bin
      - name: cni
        hostPath:
          path: /etc/cni/net.d
      - name: flannel-cfg
        configMap:
          name: kube-flannel-cfg
      - name: xtables-lock
        hostPath:
          path: /run/xtables.lock
          type: FileOrCreate

5.（关键步骤）设置 master 节点允许调度, 默认不允许调度 pod

原因：当部署单机版的 k8s 时，这个时候 master 节点是默认不允许调度 pod 。导致创建的pod一直处于pending状态 执行以下命令, 将 master 标记为可调度即可

kubectl taint nodes --all node-role.kubernetes.io/master-

至此单节点的k8s部署已经结束

6.配置Kubectl命令自动补全功能

在使用bash命令行时，在提示符下输入某个命令的前面几个字符，然后按TAB键，就会列出以这几个字符开头的命令供我们选择。不光如此，还可以进行参数补全，但只限于文件参数，当输入到参数部分时，按TAB键，就会列出以这个参数开头的文件路径供我们选择。

现在http://www.caliban.org/bash/上提供了一个名为bash-completion的软件包，通过一个复杂的脚本bash_completion实现可编程的补全程序，减少系统管理员日常维护工作，减少差错，提高工作效率。

安装
依次执行以下命令进行安装

# 安装 bash-completion
yum -y install bash-completion

# 执行 source 命令：
source /usr/share/bash-completion/bash_completion


source <(kubectl completion bash)


echo "source <(kubectl completion bash)" >> ~/.bashrc

验证
验证自动补全的效果（双击 Tab 键）：

[root@master deploy-demo]# kubectl

alpha          apply          certificate    cordon         delete         edit           get            logs           port-forward   run            top
annotate       attach         cluster-info   cp             describe       exec           help           options        proxy          scale          uncordon
api-resources  auth           completion     create         diff           explain        kustomize      patch          replace        set            version
api-versions   autoscale      config         debug          drain          expose         label          plugin         rollout        taint          wait

也可以自动补全pod等资源名称, 如以下操作， 将自动补全以 dem 开头的 pod
kubectl get pod dem + [Tab]

转载请注明出处来源