k8s集群搭建

环境准备

系统版本：CentOS Linux release 7.6.1810 (Core)
机器配置：

主机名	配置	IP
k8s-master	2c2g60G	192.168.241.130
k8s-node1	1c2g60G	192.168.241.131
k8s-node2	1c2g60G	192.168.241.132

注意：master节点CPU至少为2核

前提配置

hostname配置

https://www.cnblogs.com/zhaojiedi1992/p/zhaojiedi_linux_043_hostname.html

hostnamectl set-hostname k8s-master

[root@k8s-master ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.241.130 k8s-master
192.168.241.131 k8s-node1
192.168.241.132 k8s-node2

k8s-node1、k8s-node1同理进行配置

yum源配置

https://mirrors.cnnic.cn/help/centos/
https://blog.csdn.net/xiaojin21cen/article/details/84726193
安装清华源

sed -e 's|^mirrorlist=|#mirrorlist=|g' \
         -e 's|^#baseurl=http://mirror.centos.org|baseurl=https://mirrors.tuna.tsinghua.edu.cn|g' \
         -i.bak \
         /etc/yum.repos.d/CentOS-*.repo

安装epel源

yum install epel-release -y		;\
yum clean all					;\
yum makecache

关闭防火墙

systemctl stop firewalld		;\
systemctl disable firewalld		

关闭selinux

重启生效

sed -i '7s/SELINUX=enforcing/SELINUX=permissive/g' /etc/selinux/config

关闭swap

https://blog.csdn.net/yefun/article/details/102772368
重启生效

[root@k8s-master ~]# cat /etc/fstab 

#
# /etc/fstab
# Created by anaconda on Wed Jun 30 18:53:58 2021
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/centos-root /                       xfs     defaults        0 0
UUID=8d57733b-dec8-4af9-bc90-772ee952e5f6 /boot                   xfs     defaults        0 0
/dev/mapper/centos-home /home                   xfs     defaults        0 0
#/dev/mapper/centos-swap swap                    swap    defaults        0 0

echo vm.swappiness=0 >> /etc/sysctl.conf

网络参数配置

https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/

modprobe br_netfilter

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

sudo sysctl --system

检查MAC地址、product_uuid是否唯一

安装容器运行时

https://docs.docker.com/engine/install/centos/

yum install wget -y
wget https://download.docker.com/linux/centos/docker-ce.repo -P /etc/yum.repos.d/	;\
yum install docker-ce docker-ce-cli containerd.io -y								;\
systemctl enable docker;systemctl daemon-reload;systemctl restart docker			;\
docker run hello-world

常用命令：

删除容器：docker rm \<id>
删除镜像：docker rmi \<id>
搜索镜像：docker serach \<name>
拉取镜像：docker pull \<name>
查看镜像：docker images
查看运行中的容器：docker ps
查看所有容器：docker ps -a

cgroups配置systemd

https://kubernetes.io/docs/setup/production-environment/container-runtimes/

cat <<EOF | sudo tee /etc/docker/daemon.json
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF

systemctl enable docker;systemctl daemon-reload;systemctl restart docker

安装kubeadm、kubelet、kubectl

https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kubelet kubeadm kubectl
EOF

yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes		;\
systemctl enable --now kubelet

安装失败，请配置国内镜像源：https://www.orchome.com/10036

kubeadm init（only master）

kubeadm init初始化报错问题参考：
https://blog.csdn.net/qq_31024251/article/details/115752033
https://www.cnblogs.com/qiaoer1993/p/14504615.html
查看kubeadm所需的镜像列表

[root@k8s-master ~]# kubeadm config images list
k8s.gcr.io/kube-apiserver:v1.21.2
k8s.gcr.io/kube-controller-manager:v1.21.2
k8s.gcr.io/kube-scheduler:v1.21.2
k8s.gcr.io/kube-proxy:v1.21.2
k8s.gcr.io/pause:3.4.1
k8s.gcr.io/etcd:3.4.13-0
k8s.gcr.io/coredns/coredns:v1.8.0

配置阿里云的镜像地址进行初始化操作

kubeadm init --image-repository registry.aliyuncs.com/google_containers

将下载后的images改名，coredns需要手动下载

docker tag registry.aliyuncs.com/google_containers/kube-apiserver:v1.21.2 k8s.gcr.io/kube-apiserver:v1.21.2 ;\
docker tag registry.aliyuncs.com/google_containers/kube-proxy:v1.21.2 k8s.gcr.io/kube-proxy:v1.21.2 ;\
docker tag registry.aliyuncs.com/google_containers/kube-scheduler:v1.21.2 k8s.gcr.io/kube-scheduler:v1.21.2 ;\
docker tag registry.aliyuncs.com/google_containers/kube-controller-manager:v1.21.2 k8s.gcr.io/kube-controller-manager:v1.21.2 ;\
docker tag registry.aliyuncs.com/google_containers/etcd:3.4.13-0 k8s.gcr.io/etcd:3.4.13-0 ;\
docker tag registry.aliyuncs.com/google_containers/pause:3.4.1 k8s.gcr.io/pause:3.4.1 ;\
docker tag coredns/coredns:latest k8s.gcr.io/coredns/coredns:v1.8.0

重新初始化，kubeadm init

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.246.130:6443 --token yqhnxj.rsfhpt1istp47c3e \
	--discovery-token-ca-cert-hash sha256:3bb24fcf0b356d50df0b2ed39a61cfe2bb8d68828ecf2959d1cf813d5204a165 

配置集群，或写入/etc/profile

export KUBECONFIG=/etc/kubernetes/admin.conf

加入污点

kubectl taint nodes --all node-role.kubernetes.io/master-

kubectl常用命令

健康检查：kubectl get cs 
查看节点：kubectl get nodes
查看pod：kubectl get pods --all-namespaces
查看节点状态：kubectl describe node k8s-master
查看pod状态：...

安装weave插件（only master）

https://kubernetes.io/docs/concepts/cluster-administration/networking/#how-to-implement-the-kubernetes-networking-model
https://www.weave.works/docs/net/latest/kubernetes/kube-addon/
安装

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

删除

kubectl delete -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

安装dashboard插件（only master）

https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/
https://github.com/kubernetes/dashboard
安装

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.3.1/aio/deploy/recommended.yaml

问题：raw.githubusercontent.com无法访问？https://blog.csdn.net/u012782078/article/details/106109620
文件下载后，修改配置，实现外网访问 https://www.cnblogs.com/imstrive/p/11480424.html

---

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  type: NodePort
  ports:
    - port: 443
      targetPort: 8443
      nodePort: 30000
  selector:
    k8s-app: kubernetes-dashboard

---

安装插件

kubectl  apply -f recommended.yaml

访问https://192.168.241.130:30000
获取token

kubectl -n kube-system describe $(kubectl -n kube-system get secret -n kube-system -o name | grep namespace) | grep token

将node节点加入集群（only node）

https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/
查看token，或生成token

kubeadm token list
kubeadm token create

查看discovery-token-ca-cert-hash

openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | \
   openssl dgst -sha256 -hex | sed 's/^.* //'

加入集群

kubeadm join 192.168.241.130:6443 --token 26xa4h.mts7rn4cc33uaq5n --discovery-token-ca-cert-hash sha256:7d5dcafcc4bf5c41f7ad75066ddb3b0de7cf4af8b877f2d8b70c5c7123c10de2

重新注册

kubeadm reset

解决NotReady问题：https://blog.csdn.net/erhaiou2008/article/details/103999538

scp -r k8s-master:/etc/cni /etc/cni								;\
scp k8s-master:/opt/cni/bin/weave-ipam /opt/cni/bin/			;\
scp k8s-master:/opt/cni/bin/weave-net /opt/cni/bin/				;\
scp k8s-master:/opt/cni/bin/weave-plugin-2.8.1 /opt/cni/bin/	;\
systemctl restart kubelet										

查看kubelet服务日志

journalctl -f -u kubelet

NotReady问题解决，从节点也需手动安装kube-proxy、pause组件

成功页面