unutun21.04安装k8s v1.23.1（一）

1. 环境初始化

#关闭swap 我第一次执行这个好像失败了，导致了后面步骤的错误
swapoff -a
rm -f /swap.img
vim /etc/fstab
# /swap.img

#开启ip转发
vim /etc/sysctl.conf
net.ipv4.ip_forward=1

#查看状态
sysctl -p

2. 安装docker

2.1 安装依赖

sudo apt-get update
sudo apt-get -y install apt-transport-https ca-certificates curl software-properties-commo

2.2 安装gpg证书

curl -fsSL https://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | sudo apt-key add -

2.3 写入软件源信息

sudo add-apt-repository "deb [arch=amd64] https://mirrors.aliyun.com/docker-ce/linux/ubuntu $(lsb_release -cs) stable"

2.4 更新并安装Docker-ce

sudo apt-get -y update
apt install -y docker-ce

2.5 安装docker-compose

apt install -y docker-compose

2.6 将docker设置为开机自启

systemctl enable docker

3 安装k8s

3.1 添加证书

curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add - 

3.2 添加k8s源

sudo vim /etc/apt/sources.list.d/kubernetes.list

# 将下面的阿里源加入文件中
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main

# 也可以选择中科大的源
deb http://mirrors.ustc.edu.cn/kubernetes/apt kubernetes-xenial main

3.3 安装k8s

apt-get install -y kubelet=1.23.1-00 kubeadm=1.23.1-00 kubectl=1.23.1-00

3.4 开机启动

sudo systemctl enable kubelet && sudo systemctl start kubelet

3.5 查看所需镜像

kubeadm config images list --kubernetes-version=v1.23.1

k8s.gcr.io/kube-apiserver:v1.23.1
k8s.gcr.io/kube-controller-manager:v1.23.1
k8s.gcr.io/kube-scheduler:v1.23.1
k8s.gcr.io/kube-proxy:v1.23.1
k8s.gcr.io/pause:3.6
k8s.gcr.io/etcd:3.5.1-0
k8s.gcr.io/coredns/coredns:1.8.6

3.6 从国内的地址下载镜像

docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.23.1
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.23.1
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.23.1
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.23.1
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.6
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.5.1-0
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.8.6

3.7 为镜像重新打tag

docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.23.1 k8s.gcr.io/kube-apiserver:v1.23.1

docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.23.1 k8s.gcr.io/kube-controller-manager:v1.23.1

docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.23.1 k8s.gcr.io/kube-scheduler:v1.23.1

docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.23.1 k8s.gcr.io/kube-proxy:v1.23.1

docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.6 k8s.gcr.io/pause:3.6

docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.5.1-0 k8s.gcr.io/etcd:3.5.1-0

docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.8.6 k8s.gcr.io/coredns/coredns:1.8.6

3.8 初始化master节点

kubeadm init --kubernetes-version=v1.23.1 --pod-network-cidr=10.244.0.0/16 --service-cidr=10.96.0.0/12 --ignore-preflight-errors=Swap

出现错误：

It seems like the kubelet isn't running or healthy. The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.

查看日志

tail -100 /var/log/messages

又出问题，日志不存在

No such file or directory

/var/log/message属于系统日志，某些ubutun系统默认没开，需要手动开启

vim /etc/rsyslog.d/50-default.conf

新增：

*.info;mail.none;authpriv.none;cron.none        /var/log/messages

重启服务

systemctl restart rsyslog.service

继续查看日志：

tail -n 10 /var/log/messages

发现关键信息

“Failed to run kubelet” err=“failed to run Kubelet: running with swap on is not supported, please disable swap~ or set --fail-swap-on flag to false.”

处理：关掉swap

sudo swapoff -a

sudo vim /etc/fstab # 注释最后一行，关于swap的语句

# 重启服务器
reboot

再试试

sudo kubeadm reset

sudo kubeadm init --kubernetes-version=v1.23.1 --pod-network-cidr=10.244.0.0/16 --service-cidr=10.96.0.0/12 --ignore-preflight-errors=Swap

同样还是初始化报错，连接被拒绝，继续看日志

tail -n 10 /var/log/messages

failed to run Kubelet: misconfiguration: kubelet cgroup driver: \"systemd\" is different from docker cgroup driver: \"cgroupfs\""

意思就是docker的cgroup驱动和kubelet驱动不一致，按照网上踩坑的方案，改docker的更简单。

sudo vim /etc/docker/daemon.json 

# 新增内容（我这个文件不存在，直接新建）
{"exec-opts": ["native.cgroupdriver=systemd"]}

再次重新init，这回提示成功，按照提示配置

Your Kubernetes control-plane has initialized successfully! #初始化成功

To start using your cluster, you need to run the following as a regular user: #提示如何配置kubectl

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config 
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 10.0.0.137:6443 --token 7fmuqu.hbr7n14o7kpbx8iw \
    --discovery-token-ca-cert-hash sha256:9794706d7eadeecbc14ba8372fab500d90c624748c2c45ae212d0f32889e4071 

再次看下日志，还是又报错(暂时不管）

"Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:docker: "

3.9 按照提示配置：

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config 
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

配置完，执行kubectl get pod --all-namespaces，

提示error loading config file “/etc/kubernetes/admin.conf”: open /etc/kubernetes/admin.conf: permission denied

第一反应，使用sudo执行，但提示：

The connection to the srver localhost:8080 was refused - did you specify the right host or port

查了网上资料，解决方案主要两点：

1. 按照提示配置
2. 不要用sudo执行

思考：提示配置已经执行过了，如果不用sudo执行，会无法访问/etc下的文件。所以考虑是不是让kubectl读取配置的是，读用户名录下的文件，故而执行：

export KUBECONFIG=$HOME/.kube/config

问题解决，继续下一步。

3.10 安装pod network

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp.flannel.unprivileged created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created

这里还出现了一个幺蛾子，自己电脑无法访问 raw.githubusercontent.com ，只能在能访问的机器上下载yml内容，上传到自己阿里云服务器上，通过nginx得到一个公网的下载地址。然后在虚拟机访问自己阿里云的地址（虚拟机的unbuntu-server不能复制粘贴）

3.11 检查状态

kubectl get pods -n kube-system -l app=flannel

4 安装图形界面

kubectl apply -f https://kuboard.cn/install-script/kuboard.yaml

kubectl apply -f https://addons.kuboard.cn/metrics-server/0.3.6/metrics-server.yaml

安装metrics-server的时候，出错了，查看pod状态

kubectl get pods --all-namespaces
# 不加--namespace会找不到pod; pod和namespace都在get pods里可以看到
kubectl describe pod metrics-server-854dd89ccd-x4txb --namespace kube-system

发现问题：

node(s) had taint node-role.kubernetes.io/master that the pod didn’t tolerate

处理：允许master部署pod

kubectl taint nodes --all node-role.kubernetes.io/master-

再次查看pod状态，running ~

自此，k8s已经安装完毕，由于是虚拟机，还需要配置网络，才能在宿主机访问，查看效果。下回再更～

---

安装流程参考：
https://blog.csdn.net/professorman/article/details/118150688
问题排查主要参考：
https://www.cnblogs.com/machangwei-8/p/15759077.html
web界面参考：
https://segmentfault.com/a/1190000023092712

在整个安装（排查）过程中还查询了其他文章与官方文档等，未能完全记录地址。