k8s和docker部署详细步骤和每步原因

最新推荐文章于 2024-09-09 17:47:12 发布

20481024

最新推荐文章于 2024-09-09 17:47:12 发布

阅读量5.5k

点赞数 1

分类专栏：学习问题 java服务器文章标签： docker linux k8s k8s安装异常

本文链接：https://blog.csdn.net/zt011052/article/details/120371778

版权

学习问题同时被 2 个专栏收录

20 篇文章 1 订阅

订阅专栏

java服务器

8 篇文章 0 订阅

订阅专栏

k8s部署详细步骤

安装过程出现任何异常查看第二节异常处理

一、安装docker
https://docs.docker.com/engine/install/centos/

Uninstall old versions

 sudo yum remove docker \
                  docker-client \
                  docker-client-latest \
                  docker-common \
                  docker-latest \
                  docker-latest-logrotate \
                  docker-logrotate \
                  docker-engine

Set up the repository 配置阿里云仓库

sudo yum install -y yum-utils
sudo yum-config-manager     --add-repo     http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

Install Docker Engine

sudo yum install docker-ce docker-ce-cli containerd.io

开启启动

	chkconfig docker on
	service docker start

配置腾讯源

echo "OPTIONS='--registry-mirror=https://mirror.ccs.tencentyun.com'" >> /etc/sysconfig/docker

 systemctl daemon-reload
 service docker restart

创建私有仓库方便k8s共享

	docker volume create portainer_data
   docker run -d -p 5000:5000 -v portainer_data:/data --name myregistry2 registry:2

配置源提交

vi /etc/docker/daemon.json

{
 "insecure-registries": [ "172.21.16.6:5000"]
}
 
 systemctl daemon-reload
 service docker restart
 docker commit -m="new from zsx" -a="zsx" 7047a24235ab 127.0.0.1:5000/zhang/test:3.0

拉取

docker pull 172.21.16.6:5000/zhang/test:3.0

二、安装k8s V1.22.1
k8s官网

1. 准备

	● 每台机器 2 GB 或更多的 RAM （如果少于这个数字将会影响你应用的运行内存)
	● master 2 CPU 核或更多
	● 集群中的所有机器的网络彼此均能相互连接(公网和内网都可以)
	● 节点之中不可以有重复的主机名、MAC 地址或 product_uuid。请参见这里了解更多详细信息。
	● 开启机器上的某些端口。请参见这里 了解更多详细信息。
	● 禁用交换分区。
	
	k8s1	1c2g	172.21.16.13	node
	k8s2	1c2g	172.21.16.2	node
	k8s3	1c2g	172.21.16.15	node
	k8s4	1c2g	172.21.16.14	node
	k8s5	2c4g	172.21.16.6	master配置hosts

2. 禁用交换分区

	● 一个是性能问题，开启swap会严重影响性能（包括内存和I/O）
	● 另一个是管理问题，开启swap后通过cgroups设置的内存上限就会失效
	cgroups 是Linux内核提供的一种可以限制单个进程或者多个进程所使用资源的机制，
	可以对 cpu，内存等资源实现精细化的控制，目前越来越火的轻量级容器 Docker 
	就使用了 cgroups 提供的资源限制能力来完成cpu，内存等部分的资源控制。

sudo swapoff -a
free -m

3. 关闭防火墙

	RedHat linux操作系统中默认开启了防火墙，
	SELinux也处于启动状态，一般状态为enforing。
	致使很多服务端口默认是关闭的 ，可能造成各种外部的连接不成功

iptables -F
systemctl stop firewalld
systemctl disable firewalld
setenforce 0

4. 允许 iptables 检查桥接流量

	为了让你的 Linux 节点上的 iptables 能够正确地查看桥接流量，
	你需要确保在你的 sysctl 配置中将 net.bridge.bridge-nf-call-iptables 设置为 1。
	例如：

	cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
	br_netfilter
	EOF
	
	cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
	net.bridge.bridge-nf-call-ip6tables = 1
	net.bridge.bridge-nf-call-iptables = 1
	EOF
	sudo sysctl --system


	执行命令使修改生效。

modprobe br_netfilter
sysctl -p /etc/sysctl.d/k8s.conf

5. docker 配置
● 数据源
● [“native.cgroupdriver=systemd”] 重要

1,什么是cgroups？ cgroups(Control Groups) 是 linux 内核提供的一种机制
它可以限制、记录任务组所使用的物理资源它是内核附加在程序上的hook，使程序运行时对资源的调度触发相应的钩子,
达到资源追踪和限制资源使用的目的 2，cgroupfs是什么? docker默认的Cgroup Driver是cgroupfs
cgroupfs是cgroup为给用户提供的操作接口而开发的虚拟文件系统类型，
它和sysfs，proc类似，可以向用户展示cgroup的hierarchy，通知kernel用户对cgroup改动
对cgroup的查询和修改只能通过cgroupfs文件系统来进行 3,为什么要修改为使用systemd? Kubernetes 推荐使用
systemd 来代替 cgroupfs 因为systemd是Kubernetes自带的cgroup管理器,
负责为每个进程分配cgroups, 但docker的cgroup
driver默认是cgroupfs,这样就同时运行有两个cgroup控制管理器, 当资源有压力的情况时,有可能出现不稳定的情况
如果不修改配置，会在kubeadm init时有提示:

[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:8/healthz": dial tcp [::1]:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:8/healthz": dial tcp [::1]:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:8/healthz": dial tcp [::1]:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:8/healthz": dial tcp [::1]:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:8/healthz": dial tcp [::1]:10248: connect: connection refused.

	Unfortunately, an error has occurred:
		timed out waiting for the condition

	This error is likely caused by:
		- The kubelet is not running
		- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

	If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
		- 'systemctl status kubelet'
		- 'journalctl -xeu kubelet'

journalctl -xeu kubelet 给出详细提示 error: failed to run Kubelet: failed to
create kubelet: misconfiguration: kubelet cgroup driver: “systemd” is
different from docker cgroup driver: "cgroupfs

cat > /etc/docker/daemon.json <<EOF
{
  "registry-mirrors": ["https://mirror.ccs.tencentyun.com"]
,
"insecure-registries": [ "172.21.16.6:5000"],
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ],
  "data-root": "/data/docker"
}
EOF

systemctl daemon-reload && systemctl restart docker

6. 下载所需要的镜像
● 查看镜像

[root@VM-16-14-centos ~]# kubeadm config images list
k8s.gcr.io/kube-apiserver:v1.22.1
k8s.gcr.io/kube-controller-manager:v1.22.1
k8s.gcr.io/kube-scheduler:v1.22.1 k8s.gcr.io/kube-proxy:v1.22.1
k8s.gcr.io/pause:3.5 k8s.gcr.io/etcd:3.5.0-0
k8s.gcr.io/coredns/coredns:v1.8.4

● 手动下载自动下载云服务器访问不到地址
在这里插入图片描述

set -o errexit
set -o nounset
set -o pipefail


##这是原始仓库名，最后需要改名成这个
GCR_URL=k8s.gcr.io

##这里就是写你要使用的仓库
DOCKERHUB_URL=gotok8s

##这里是镜像列表，新版本要把coredns改成coredns/coredns
images=(
kube-proxy:v1.22.1
kube-scheduler:v1.22.1
kube-controller-manager:v1.22.1
kube-apiserver:v1.22.1
pause:3.5
etcd:3.5.0-0
coredns:v1.8.4
)

##这里是拉取和改名的循环语句
for imageName in ${images[@]} ; do
  docker pull $DOCKERHUB_URL/$imageName
  docker tag $DOCKERHUB_URL/$imageName $GCR_URL/$imageName
  docker rmi $DOCKERHUB_URL/$imageName
done

7. 安装 kubeadm、kubelet 和 kubectl

yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
systemctl enable --now kubelet#开启启动

和第5步docker保持一致 cgroup = systemd

● vi /etc/sysconfig/kubelet

改为如下参数
KUBELET_EXTRA_ARGS=--cgroup-driver=systemd

9. mast 节点初始化

       274  2021-09-03 00:20:00 kubeadm init
  -----下面详细输出
  275  2021-09-03 00:22:57 mkdir -p $HOME/.kube
  276  2021-09-03 00:23:05 sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  277  2021-09-03 00:23:13 sudo chown $(id -u):$(id -g) $HOME/.kube/config


[init] Using Kubernetes version: v1.22.1
[preflight] Running pre-flight checks
	[WARNING FileExisting-tc]: tc not found in system path
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local vm-16-6-centos] and IPs [10.96.0.1 172.21.16.6]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost vm-16-6-centos] and IPs [172.21.16.6 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost vm-16-6-centos] and IPs [172.21.16.6 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 8.004485 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.22" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node vm-16-6-centos as control-plane by adding the labels: [node-role.kubernetes.io/master(deprecated) node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node vm-16-6-centos as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: re8oxx.g4qdeaf4y4gqqw7z
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 172.21.16.6:6443 --token re8oxx.g4qdfaf4y4gqqw7z \
	--discovery-token-ca-cert-hash sha256:001edcc759db8318aa7e2c6e31d42542714f0a0e2659ac69e347b9aa30353d88
  node使用

10. node join master

kubeadm join 172.21.16.6:6443 --token re8oxx.g4qdfaf4y4gqqw7z \
	--discovery-token-ca-cert-hash sha256:001edcc759db8318aa7e2c6e31d42542714f0a0e2659ac69e347b9aa30353d88

11. master 查看node状态和 pod状态

kubectl  get nodes
kubectl  get pods --namespace=kube-system
kubectl get pods -n kube-system -o wide

在这里插入图片描述

12. 安装 Pod 网络附加组件

官网解释你必须部署一个基于 Pod 网络插件的容器网络接口 (CNI)，以便你的 Pod 可以相互通信。在安装网络之前，集群 DNS
(CoreDNS) 将不会启动。 ● 注意你的 Pod 网络不得与任何主机网络重叠：如果有重叠，你很可能会遇到问题。
（如果你发现网络插件的首选 Pod 网络与某些主机网络之间存在冲突，则应考虑使用一个合适的 CIDR 块来代替，然后在执行
kubeadm init 时使用 --pod-network-cidr 参数并在你的网络插件的 YAML 中替换它）。 ●
默认情况下，kubeadm 将集群设置为使用和强制使用 RBAC（基于角色的访问控制）。确保你的 Pod 网络插件支持
RBAC，以及用于部署它的 manifests 也是如此。 ● 如果要为集群使用 IPv6（双协议栈或仅单协议栈 IPv6 网络），
请确保你的Pod网络插件支持 IPv6。 IPv6 支持已在 CNI v0.6.0 版本中添加。

wget  https://docs.projectcalico.org/manifests/calico.yaml
kubectl apply -f calico.yaml
kubectl get pods --all-namespaces #检查 CoreDNS Pod 是否 Running

三、操作中非个人因素异常
1. kubeadm 执行命令命令错误后再执行异常
解决：kubeadm reset

  ![在这里插入图片描述](https://img-blog.csdnimg.cn/907bc5a3920c4c04a2237a872f0b6f5f.png?x-oss-process=image/watermark,type_ZHJvaWRzYW5zZmFsbGJhY2s,shadow_50,text_Q1NETiBAMjA0ODEwMjQ=,size_20,color_FFFFFF,t_70,g_se,x_16)

2. kubectl get cs 查看集群状态 unhealth 问题端口未开
解决:

vi /etc/kubernetes/manifests/kube-scheduler.yam    #port=0那行注释

在这里插入图片描述

3. 节点 kubectl get cs 查看集群状态 unhealth refused

[root@VM-16-14-centos ~]# kubectl get pods 
The connection to the server localhost:8080 was refused - did you specify the right host or port?

	解决

在这里插入图片描述

then 所有node

scp 172.21.16.6:/etc/kubernetes/admin.conf /etc/kubernetes/admin.conf

echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile

source ~/.bash_profile

节点 kubectl get nodes NotReady

● 有问题的节点kubelet的日志

[root@VM-16-6-centos manifests]# journalctl -f -u kubelet
-- Logs begin at Thu 2021-09-02 11:18:04 CST. --
Sep 03 15:40:43 VM-16-6-centos kubelet[832429]: I0903 15:40:43.426645  832429 cni.go:239] "Unable to update cni config" err="no networks found in /etc/cni/net.d"
Sep 03 15:40:44 VM-16-6-centos kubelet[832429]: E0903 15:40:44.159076  832429 kubelet.go:2332] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized"

解决：参考安装的第12安装 Pod 网络附加组件然后稍等一会查看集群状态

wget  https://docs.projectcalico.org/manifests/calico.yaml
kubectl apply -f calico.yaml
kubectl get pods --all-namespaces #检查 CoreDNS Pod 是否 Running

20481024

关注

1
点赞
踩
19

收藏

觉得还不错? 一键收藏
0
评论
复制链接

分享到 QQ

分享到新浪微博

扫一扫

专栏目录