Docker方式部署Kubernetes集群

最新推荐文章于 2024-10-06 17:55:58 发布

小白lhd

最新推荐文章于 2024-10-06 17:55:58 发布

阅读量1k

点赞数 10

文章标签： kubernetes docker 容器

本文链接：https://blog.csdn.net/m0_72656276/article/details/142714124

版权

kubernetes部署有多种方式，目前主流的方式有kubeadm、minikube、二进制，本文介绍以docker方式部署k8s集群

minikube：一个用于快速搭建单节点kubernetes的工具
kubeadm：一个用于快速搭建kubernetes集群的工具
二进制包：从官网下载每个组件的二进制包，依次去安装，此方式对于理解kubernetes组件更加有效

k8s部署环境说明：本实验系统使用Red Hat Enterprise Linux 9 64 位

主机名	IP地址	角色	配置
k8s-master	192.168.182.10	k8s集群控制节点	4颗CPU3G内存20G硬盘
k8s-node1	192.168.182.11	k8s集群工作节点	4颗CPU3G内存20G硬盘
k8s- node2	192.168.182.12	k8s集群工作节点	4颗CPU3G内存20G硬盘
docker-hub	192.168.182.100	harbor仓库	4颗CPU3G内存20G硬盘

此实验前提需要搭建一个harbor仓库

Docker容器的部署

配置软件仓库：

[root@docker-hub ~]# vim /etc/yum.repos.d/docker-ce.repo

[docker-ce]
name=docker-ce
baseurl=https://mirrors.aliyun.com/docker-ce/linux/rhel/9/x86_64/stable
gpgcheck=0

安装docker-ce并启动服务：

[root@docker-hub ~]# yum install docker-ce -y

[root@docker-hub ~]# systemctl enable --now docker.service

harbor仓库的搭建

给镜像仓库加密生成证书

[root@docker-hub ~]# vim /etc/docker/daemon.json #删除文件里的内容

[root@docker-hub ~]# systemctl restart docker.service

[root@docker-hub ~]# mkdir certs

[root@docker-hub certs]# vim /etc/hosts

192.168.182.100 reg.lhd.com

[root@docker-node1 ~]# openssl req -newkey rsa:4096 -nodes -sha256 -keyout certs/lhd.com.key -addext "subjectAltName = DNS:reg.lhd.com" -x509 -days 365 -out certs/lhd.com.crt

Generating a RSA private key

.....................++++

..............................................................................++++

writing new private key to 'certs/lhd.com.key'

-----

You are about to be asked to enter information that will be incorporated

into your certificate request.

What you are about to enter is what is called a Distinguished Name or a DN.

There are quite a few fields but you can leave some blank

For some fields there will be a default value,

If you enter '.', the field will be left blank.

-----

Country Name (2 letter code) [AU]:CN

State or Province Name (full name) [Some-State]:shaanx

Locality Name (eg, city) []:xian

Organization Name (eg, company) [Internet Widgits Pty Ltd]:docker

Organizational Unit Name (eg, section) []:registry

Common Name (e.g. server FQDN or YOUR name) []:reg.lhd.com

Email Address []:admin@lhd.com

[root@docker-hub ~]# mkdir -p /etc/docker/certs.d/reg.lhd.com

[root@docker-hub ~]# cp /root/certs/lhd.com.crt /etc/docker/certs.d/reg.lhd.com/ca.crt

[root@docker-hub ~]# ls /etc/docker/certs.d/reg.lhd.com/

ca.crt

[root@docker-hub ~]# systemctl restart docker.service

部署harbor

[root@docker-hub ~]# tar zxf harbor-offline-installer-v2.5.4.tgz

[root@docker-hub ~]# cd harbor/

[root@docker-hub harbor]# ls

common.sh harbor.v2.5.4.tar.gz harbor.yml.tmpl install.sh LICENSE prepare

[root@docker-hub harbor]# cp harbor.yml.tmpl harbor.yml

[root@docker-hub harbor]# vim harbor.yml

修改以下几条参数

hostname: reg.lhd.com

certificate: /data/certs/lhd.com.crt #你自己的证书位置和密钥位置

private_key: /data/certs/lhd.com.key

harbor_admin_password: 123

建立目录拷贝生成的证书

[root@docker-hub harbor]# mkdir /data

[root@docker-hub harbor]# cp /root/certs/ -r /data/

[root@docker-hub harbor]# ll /data/certs/

total 8

-rw-r--r--. 1 root root 2147 Sep 1 16:59 lhd.com.crt

-rw-------. 1 root root 3272 Sep 1 16:59 lhd.com.key

自动开启容器

[root@docker-hub harbor]# ./install.sh --with-chartmuseum

登陆并认证

[root@docker-hub harbor]# vim /etc/docker/daemon.json

{

"registry-mirrors": ["https://reg.lhd.com"]

}

[root@docker-hub harbor]# docker login reg.lhd.com

最后可拉取或上传镜像

部署k8s集群

k8s-all 代表所有节点

所有节点禁用selinux和防火墙

所有节点同步时间和解析

所有节点安装docker-ce
所有节点禁用swap，注意注释掉/etc/fstab文件中的定义

所有阶段禁用selinux和防火墙并做hosts解析

[root@k8s-all ~]# systemctl mask dev-nvme0n1p3.swap

[root@k8s-all ~]# swapoff -a

[root@k8s-all ~]# vim /etc/fstab

#/dev/mapper/rhel-swap swap swap defaults 0 0

[root@k8s-all ~]# vim /etc/hosts

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.182.100 reg.lhd.com
192.168.182.10 k8s-master
192.168.182.11 k8s-node1
192.168.182.12 k8s-node2

所有阶段复制harbor仓库中的证书并启动docker

[root@k8s-all ~]# ls -l /etc/docker/certs.d/reg.lhd.com/
总用量 4
-rw-r--r-- 1 root root 2134 9月 28 12:51 ca.crt
[root@k8s-master ~]# systemctl enable --now docker

[root@k8s-master ~]# docker login reg.lhd.com
Authenticating with existing credentials...
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credential-stores

Login Succeeded

[root@k8s-all ~]# docker info
Client: Docker Engine - Community
Version: 27.3.1
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.17.1
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.29.7
Path: /usr/libexec/docker/cli-plugins/docker-compose

Server:
Containers: 34
Running: 32
Paused: 0
Stopped: 2
Images: 15
Server Version: 27.3.1
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c
runc version: v1.1.14-0-g2c9f560
init version: de40ad0
Security Options:
seccomp
Profile: builtin
Kernel Version: 5.14.0-362.8.1.el9_3.x86_64
Operating System: Red Hat Enterprise Linux 9.3 (Plow)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 2.673GiB
Name: k8s-master
ID: 96c3fcb0-eb3b-452d-a208-34f1da4abb41
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
127.0.0.0/8
Registry Mirrors:
https://reg.lhd.com/
Live Restore Enabled: false

WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

所有阶段安装部署k8s工具

#部署软件仓库，添加K8S源

[root@k8s-all ~]# vim /etc/yum.repos.d/k8s.repo

[k8s]
name=k8s
baseurl=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.30/rpmgpgcheck=0
#安装软件
[root@k8s-all ~]# dnf install kubelet-1.30.0 kubeadm-1.30.0 kubect1-1.30.0 -y

所有节点设置kubectl命令补齐功能

[root@k8s-all ~]# dnf install bash-completion -y

[root@k8s-all ~]# echo "source <(kubectl completion bash)" >> ~/.bashrc

[root@k8s-all ~]# source~/.bashrc

所有节点安装cri-docker

[root@k8s-all ~]# dnf install libcgroup-0.41-19.e18.x86_64.rpm \

> cri-dockerd-0.3.14-3.e18.x86_64.rpm -y
[root@k8s-all ~l# vim /lib/systemd/system/cri-docker.service

[unit]
Description=CRI Interface for Docker Application container Engine

Documentation=https://docs.mirantis.com

After=network-online.target firewalld.service docker.service

Wants=network-online.target

Requires=cri-docker.socket
[Service]
Type=notify
#指定网络插件名称及基础容器镜像
Execstart=/usr/bin/cri-dockerd --container-runtime-endpoint fd:// --network-blugin=cni --pod-infra-container-image=reg.lhd.com/k8s/pause:3.9
ExecReload=/bin/kill -S HUP $MAINPID

TimeoutSec=0
RestartSec=2
Restart=always

[root@k8s-all ~]# systemctl daemon-reload

[root@k8s-all ~]# systemctl start cri-docker

[root@k8s-all ~]# ll /var/run/cri-dockerd.sock

在master控制节点拉取k8s所需镜像

#拉取k8s集群所需要的镜像
[root@k8s-master ~]# kubeadm config images pull \

--image-repository registry.aliyuncs.com/google_containers \

--kubernetes-version v1.30.0 \
--cri-socket=unix:///var/run/cri-dockerd.sock
#上传镜像到harbor仓库
[root@k8s-master ~]# docker images | awk '/google/{ print $1":"$2} |

awk -F "/" '{system("docker tag "$0" reg.lhd.com/k8s/" $3)}'
[root@k8s-master ~]# docker images | awk '/k8s/{system("docker push " $1":"$2)}'

集群初始化

#启动kubelet服务
[root@k8s-master ~]# systemctl status kubelet.service
#执行初始化命令
[root@k8s-master ~]# kubeadm init --pod-network-cidr=10.244.0.0/16 \

--image-repository reg.timinglee.org/k8s \

--kubernetes-version v1.30.0 \
--cri-socket=unix:///var/run/cri-dockerd.sock
#指定集群配置文件变量
[root@k8s-master ~]# echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile

在此阶段如果生成的集群token找不到了可以重新生成

[root@k8s-master ~]# kubeadm token create --print-join-commandkubeadm join 192.168.182.100:6443 --token 5hwptm.zwn7epa6pvatbpwf

-- discovery-token-ca-cert-hash
sha256:52f1a83b70ffc8744db5570288ab51987ef2b563bf906ba4244a300f61e9db23

安装flannel网络插件

#下载flannel的yaml部署文件

[root@k8s-master ~]# wget https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
#现在镜像:
[root@k8s-master ~]# docker pull docker.io/flannel/flannel:v0.25.5

[root@k8s-master ~]# docker docker.io/flannel/flannel-cni-plugin:v1.5.1-flannel1
#上传镜像到仓库
[root@k8s-master ~]# docker tag flannel/flannel:v0.25.5 reg.lhd.com/flannel/flannel:v0.25.5

[root@k8s-master ~]# docker push reg.lhd.com/flannel/flannel:v0.25.5
[root@k8s-master ~]# docker tag flannel/flanne]-cni-plugin:v1.5.1 flannel reg.lhd.com/flannel/flannel-cni-plugin:v1.5.1-flannel1

[root@k8s-master ~]# docker push reg.lhd.com/flannel/flannel-cni-plugin:v1.5.1-flannel1

#编辑kube-flannel.yml 修改镜像下载位置

[root@k8s-master ~]# vim kube-flannel.yml
#需要修改以下几行
[root@k8s-master ~]# grep -n image kube- flannel.yml

146: image: reg.lhd.com/flannel/flannel:v0.25.5

173: image: reg.lhd.com/flannel/flannel-cni-plugin:v1.5.1-flannel1
184: image: reg.lhd.com/flannel/flannel:v0.25.5
#安装flannel网络插件
[root@k8s-master ~]# kubectl apply -f kube-flannel.yml

信息确认完毕后可加入集群

[root@k8s-node1 &2 ~]# kubeadm join 192.168.182.100:6443 --token 5hwptm.zwn7epa6pvatbpwf --discovery-token-ca-cert-hash sha256:52f1a83b70ffc8744db5570288ab51987ef2b563bf906ba4244a300f61e9db23 --cri-socket=unix:///var/run/cri-dockerd.sock

在master阶段中查看所有node的状态

[root@k8s-master ~]# kubectl get nodes

NAME STATUS ROLES   AGE VERSION
k8s-master.timinglee.org Ready control-plane 98m v1.30.0

k8s-nodel.timinglee.org Ready   <none>   21m v1.30.0

k8s-node2.timinglee.org   Ready   <none>   21m v1.30.0
所有阶段的STATUS状态为Ready状态，那么恭喜你，你的kubernetes就装好了

测试集群运行状态

#建立一个pod
[root@k8s-master ~]# kubect run test --image nginx
#查看pod状态
[root@k8s-master ~]# kubectl get pods

NAME READY STATUS RESTARTS AGE

test 1/1 Running 0 6m29s
#删除pod
root@k8s-master ~]# kubectl delete pod