本篇文章带来制作yum仓库的镜像实战项目,要求利用到初始化容器技术,可用性探针检查livessProbe,rc资源部署2个副本,svc提供统一的访问入口,在K8S内部启动一个centos进行访问测试,话不多说,直接开始!
手动测试centos镜像制作yum仓库
手动走一遍centos镜像制作yum仓库的流程,可以参考之前写过的文章
【运维知识进阶篇】一键部署yum本地仓库_我是koten的博客-CSDN博客
准备rpm包,将之前做的yum仓库打包解压到232主机上
[root@Ansible share]# scp -r packages/ 10.0.0.232:/
制作yum仓库的容器
[root@Worker232 ~]# docker run -id centos:7.9.2009
[root@Worker232 ~]# docker ps -l
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6694b3fe2ec5 centos:7.9.2009 "/bin/bash" 4 seconds ago Up 3 seconds intelligent_allen
[root@Worker232 ~]# docker cp packages 6694b3fe2ec5:/
[root@Worker232 ~]# docker exec -it 6694b3fe2ec5 bash
[root@6694b3fe2ec5 /]# yum -y install createrepo
[root@6694b3fe2ec5 /]# createrepo packages/
Spawning worker 0 with 2035 pkgs
Spawning worker 1 with 2035 pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complet
安装配置nginx
[root@6694b3fe2ec5 /]# curl -so /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
[root@6694b3fe2ec5 /]# curl -so /etc/yum.repos.d/epel.repo https://mirrors.aliyun.com/repo/epel-7.repo
[root@6694b3fe2ec5 /]# yum -y install nginx
cat > /etc/nginx/conf.d/yum.conf <<EOF
server {
listen 80 default_server;
root /packages;
index index.html index.htm;
autoindex on;
autoindex_exact_size on;
autoindex_localtime on;
}
EOF
[root@6694b3fe2ec5 /]# chown -R nginx:nginx packages/
[root@6694b3fe2ec5 /]# nginx
[root@6694b3fe2ec5 /]# curl -s 127.0.0.1|head
<html>
<head><title>Index of /</title></head>
<body>
<h1>Index of /</h1><hr><pre><a href="../">../</a>
<a href="repodata/">repodata/</a> 19-Jun-2023 16:53 -
<a href="389-ds-base-1.3.10.2-6.el7.x86_64.rpm">389-ds-base-1.3.10.2-6.el7.x86_64.rpm</a> 14-Oct-2020 18:46 1818404
<a href="389-ds-base-libs-1.3.10.2-6.el7.x86_64.rpm">389-ds-base-libs-1.3.10.2-6.el7.x86_64.rpm</a> 14-Oct-2020 18:46 730568
<a href="ElectricFence-2.2.2-39.el7.x86_64.rpm">ElectricFence-2.2.2-39.el7.x86_64.rpm</a> 04-Jul-2014 01:11 36116
<a href="GConf2-3.2.6-8.el7.x86_64.rpm">GConf2-3.2.6-8.el7.x86_64.rpm</a> 04-Jul-2014 01:26 1047864
<a href="GeoIP-1.5.0-14.el7.x86_64.rpm">GeoIP-1.5.0-14.el7.x86_64.rpm</a> 22-Aug-2019 21:24 1527972
配置yum源,我就配置在yum仓库上了,下面实操再用k8s内部pod测试
#忽略下yum仓库不可用的错误
[root@6694b3fe2ec5 /]# yum-config-manager --save --setopt=local.skip_if_unavailable=true
[root@6694b3fe2ec5 /]# cat /etc/yum.repos.d/local.repo
[local]
name=local
baseurl=http://127.0.0.1/packages
enabled=1
[root@6694b3fe2ec5 /]# yum -y install tree
[root@6694b3fe2ec5 /]# mkdir test
[root@6694b3fe2ec5 /]# tree test/
test/
0 directories, 0 files
编写yaml文件(初始化容器技术+可用性探针检查)
我们在此使用的初始化容器技术是在初始化容器的过程中,把rpm包通过nginx在web页面上展示出来,此时业务容器只需要拿到展示的结果,不需要执行初始化的步骤。
此处的重点在于整理好,什么操作需要在初始化容器做,什么操作需要在创建镜像的时候就做,以我要做的yum仓库为例。
操作步骤 | 创建镜像时 | 初始化容器 |
安装createrepo | √ | |
将目录创建成yum仓库 | √ | |
添加阿里云的yum源 | √ | |
更改nginx配置文件 | √ | |
更改rpm目录的属主属组 | √ |
1、先将基础镜像做好,这里我用dockerfile的形式去做
[root@Master231 rc]# cat dockerfile
FROM harbor.koten.com/koten-linux/centos:7
RUN sed -e 's|^mirrorlist=|#mirrorlist=|g' \
-e 's|^#baseurl=http://mirror.centos.org/centos|baseurl=https://mirrors.tuna.tsinghua.edu.cn/centos|g' \
-i /etc/yum.repos.d/CentOS-*.repo && \
yum -y install epel-release && \
sed -e 's!^metalink=!#metalink=!g' \
-e 's!^#baseurl=!baseurl=!g' \
-e 's!https\?://download\.fedoraproject\.org/pub/epel!https://mirrors.tuna.tsinghua.edu.cn/epel!g' \
-e 's!https\?://download\.example/pub/epel!https://mirrors.tuna.tsinghua.edu.cn/epel!g' \
-i /etc/yum.repos.d/epel*.repo && \
yum -y install nginx createrepo
COPY yum.conf /etc/nginx/conf.d/yum.conf
CMD ["nginx","-g","daemon off;"]
[root@Master231 rc]# docker build -t harbor.koten.com/koten-linux/yumrepo:v1.0 .
[root@Master231 rc]# docker push harbor.koten.com/koten-linux/yumrepo:v1.0
2、使用初始化容器技术编写yaml文件
[root@Master231 rc]# cat 04-rc-yum-init.yaml
apiVersion: v1
kind: Pod
metadata:
name: yum-initcontainers
spec:
nodeName: worker232
# 定义存储卷
volumes:
- name: data01
hostPath:
path: /packages
# 初始化容器,仅在Pod第一次创建时触发,即只触发一次。
initContainers:
- name: init
image: harbor.koten.com/koten-linux/yumrepo:v1.0
imagePullPolicy: IfNotPresent
command: ["/bin/sh","+x","/packages/deploy_yum.sh"]
volumeMounts:
- name: data01
mountPath: /packages
# 业务容器
containers:
- name: web
image: harbor.koten.com/koten-linux/yumrepo:v1.0
volumeMounts:
- name: data01
mountPath: /packages
3、给yaml文件增加可用性探针检查
可用性检查有三种判断方式,exec,httpget,tcpsocket,我们判断在web页面上能否看到yum仓库即可,选择第一种判断方式,我写了一个命令,如果能过滤出来,该返回结果等于0,就是正常可用的
curl 127.0.0.1 -s | grep repodata
将该命令加入yaml文件,并附带相关参数
[root@Master231 rc]# cat 04-rc-yum-init.yaml
apiVersion: v1
kind: Pod
metadata:
name: yum-initcontainers
spec:
nodeName: worker232
# 定义存储卷
volumes:
- name: data01
hostPath:
path: /packages
# 初始化容器,仅在Pod第一次创建时触发,即只触发一次。
initContainers:
- name: init
image: harbor.koten.com/koten-linux/yumrepo:v1.0
imagePullPolicy: IfNotPresent
command: ["/bin/sh","+x","/packages/deploy_yum.sh"]
volumeMounts:
- name: data01
mountPath: /packages
# 业务容器
containers:
- name: web
image: harbor.koten.com/koten-linux/yumrepo:v1.0
volumeMounts:
- name: data01
mountPath: /packages
readinessProbe:
# 使用exec的方式去做健康检查
exec:
# 自定义检查的命令
command:
- /bin/sh
- -c
- curl 127.0.0.1 -s | grep repodata
# 检测服务失败次数的累加值,默认值是3次,最小值是1。当检测服务成功后,该值会被重置!
failureThreshold: 3
# 指定多久之后进行健康状态检查,即此时间段内检测服务失败并不会对failureThreshold进行计数。
initialDelaySeconds: 15
# 指定探针检测的频率,默认是10s,最小值为1.
periodSeconds: 1
# 检测服务成功次数的累加值,默认值为1次,最小值1.
successThreshold: 1
# 一次检测周期超时的秒数,默认值是1秒,最小值为1.
timeoutSeconds: 1
4、测试运行
[root@Master231 rc]# kubectl apply -f 04-rc-yum-init.yaml
[root@Master231 rc]# kubectl get po
NAME READY STATUS RESTARTS AGE
yum-initcontainers 1/1 Running 0 3m24s
RC资源部署
把pod类型的yaml文件改成rc类型即可
[root@Master231 rc]# cat 04-rc-yum-init.yaml
apiVersion: v1
kind: ReplicationController
metadata:
name: yumrepos
spec:
replicas: 2
template:
metadata:
labels:
apps: yumrepo
spec:
nodeName: worker232
# 定义存储卷
volumes:
- name: data01
hostPath:
path: /packages
# 初始化容器,仅在Pod第一次创建时触发,即只触发一次。
initContainers:
- name: init
image: harbor.koten.com/koten-linux/yumrepo:v1.0
imagePullPolicy: IfNotPresent
command: ["/bin/sh","+x","/packages/deploy_yum.sh"]
volumeMounts:
- name: data01
mountPath: /packages
# 业务容器
containers:
- name: web
image: harbor.koten.com/koten-linux/yumrepo:v1.0
volumeMounts:
- name: data01
mountPath: /packages
readinessProbe:
# 使用exec的方式去做健康检查
exec:
# 自定义检查的命令
command:
- /bin/sh
- -c
- curl 127.0.0.1 -s | grep repodata
# 检测服务失败次数的累加值,默认值是3次,最小值是1。当检测服务成功后,该值会被重置!
failureThreshold: 3
# 指定多久之后进行健康状态检查,即此时间段内检测服务失败并不会对failureThreshold进行计数。
initialDelaySeconds: 15
# 指定探针检测的频率,默认是10s,最小值为1.
periodSeconds: 1
# 检测服务成功次数的累加值,默认值为1次,最小值1.
successThreshold: 1
# 一次检测周期超时的秒数,默认值是1秒,最小值为1.
timeoutSeconds: 1
测试运行
[root@Master231 rc]# kubectl apply -f 04-rc-yum-init.yaml
replicationcontroller/yumrepos created
[root@Master231 rc]# kubectl get po
NAME READY STATUS RESTARTS AGE
yumrepos-9b2jl 1/1 Running 0 55s
yumrepos-qkwmh 1/1 Running 0 55s
svc提供统一访问
[root@Master231 svc]# cat 09-svc-yum-repos.yaml
apiVersion: v1
kind: Service
metadata:
name: yum-repos
spec:
# 基于标签选择器关联Pod
selector:
apps: yumrepo
# 配置端口映射
ports:
# 指定Service服务本身的端口号
- port: 8888
# 后端Pod提供服务的端口号
targetPort: 80
[root@Master231 svc]# kubectl apply -f 09-svc-yum-repos.yaml
service/yum-repos configured
[root@Master231 svc]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.200.0.1 <none> 443/TCP 5h
yum-repos ClusterIP 10.200.146.33 <none> 8888/TCP 40s
[root@Master231 svc]# kubectl get ep
NAME ENDPOINTS AGE
kubernetes 10.0.0.231:6443 5h
yum-repos 10.100.1.11:80,10.100.1.12:80 45s
[root@Master231 svc]# curl 10.200.146.33:8888 -s | grep repodata
<a href="repodata/">repodata/</a> 19-Feb-2024 06:39 -
K8s内部测试使用yum仓库
随便创建一个pod,然后进去后手动修改yum源,看看能不能在自己创建的yum仓库下载东西,后续优化的话可以在部署pod时候的command中去自动修改。
[root@Master231 pod]# cat 39-pod-centos.yaml
apiVersion: v1
kind: Pod
metadata:
name: linux
spec:
containers:
- name: web
image: harbor.koten.com/koten-linux/centos:7
command:
- tail
- -f
- /etc/hosts
[root@Master231 pod]# kubectl apply -f 39-pod-centos.yaml
pod/linux created
[root@Master231 pod]# kubectl get po
NAME READY STATUS RESTARTS AGE
linux 1/1 Running 0 3s
yumrepos-9b2jl 1/1 Running 0 15m
yumrepos-qkwmh 1/1 Running 0 15m
[root@Master231 pod]# kubectl exec -it linux bash
[root@linux ~]# mv /etc/yum.repos.d/* /opt/
[root@linux ~]# cat /etc/yum.repos.d/local.repo
[local]
name=local
baseurl=http://yum-repos.default:8888/packages
enabled=1
gpgcheck=0
[root@linux ~]# yum clean all
[root@linux ~]# yum makecache
[root@linux ~]# yum repolist
Loaded plugins: fastestmirror, ovl
Loading mirror speeds from cached hostfile
repo id repo name status
local local 4070
repolist: 4070
测试下载,走的是local源,下载速度非常快。
[root@linux ~]# yum -y install tree
Loaded plugins: fastestmirror, ovl
Loading mirror speeds from cached hostfile
Resolving Dependencies
--> Running transaction check
---> Package tree.x86_64 0:1.6.0-10.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
======================================================================================================================================
Package Arch Version Repository Size
======================================================================================================================================
Installing:
tree x86_64 1.6.0-10.el7 local 46 k
Transaction Summary
======================================================================================================================================
Install 1 Package
Total download size: 46 k
Installed size: 87 k
Downloading packages:
tree-1.6.0-10.el7.x86_64.rpm | 46 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : tree-1.6.0-10.el7.x86_64 1/1
Verifying : tree-1.6.0-10.el7.x86_64 1/1
Installed:
tree.x86_64 0:1.6.0-10.el7
Complete!
[root@linux ~]# tree
.
`-- anaconda-ks.cfg
0 directories, 1 file
至此,大功告成!
我是koten,10年运维经验,持续分享技术干货,感谢大家的阅读和关注!