nfs-subdir-external-provisioner简介
nfs-subdir-external-provisioner可动态为kubernetes提供pv卷,是Kubernetes的简易NFS的外部provisioner,本身不提供NFS,需要现有的NFS服务器提供存储。持久卷目录的命名规则为: ${namespace}-${pvcName}-${pvName}
。
项目地址:https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner
搭建NFS服务器
集群外准备一个节点10.0.0.5
作为NFS server服务器,这里作为测试,使用docker部署NFS服务器:
docker run -d --name nfs-server \
--privileged \
--restart always \
-p 2049:2049 \
-v /nfs-share:/nfs-share \
-e SHARED_DIRECTORY=/nfs-share \
itsthenetwork/nfs-server-alpine:latest
或者手动方式部署nfs服务器:
yum -y install nfs-utils
mkdir -p /nfs-share
chmod -R 777 /nfs-share
cat >> /etc/exports << EOF
/nfs-share *(rw,no_root_squash,sync)
EOF
exportfs -arv
exportfs -s
systemctl enable --now rpcbind
systemctl enable --now nfs-server
kubernetes集群所有节点需要安装nfs客户端
yum install -y nfs-utils
作为准备工作,我们已经搭建了一个 NFS 服务器,共享目录为 /nfs-share。下面部署nfs-subdir-external-provisioner。
helm部署nfs-subdir-external-provisioner
helm hub地址:https://artifacthub.io/packages/helm/nfs-subdir-external-provisioner/nfs-subdir-external-provisioner
添加helm仓库
helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/
使用helm部署,镜像问题需自行解决,这里使用dockerhub搜索到的镜像:
helm install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \
--namespace=nfs-provisioner \
--create-namespace \
--set image.repository=willdockerhub/nfs-subdir-external-provisioner \
--set image.tag=v4.0.2 \
--set replicaCount=2 \
--set storageClass.name=nfs-client \
--set storageClass.defaultClass=true \
--set nfs.server=10.0.0.5 \
--set nfs.path=/
注意:如果nfs server使用docker部署,在nfsv4中nfs.path
直接使用/
即可,手动部署nfs server时nfs.path
指定为/nfs-share
。
查看创建的pods
[root@master ~]# kubectl -n nfs-provisioner get pods
NAME READY STATUS RESTARTS AGE
nfs-subdir-external-provisioner-855665bf6c-c7g4f 1/1 Running 0 19m
nfs-subdir-external-provisioner-855665bf6c-fn67g 1/1 Running 0 164m
查看创建的storageclass
[root@master ~]# kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
nfs-client (default) cluster.local/nfs-subdir-external-provisioner Delete Immediate true 166m
从NFS动态申请PV
示例1:创建一个wordpress应用进行验证,其中wordpress以及mariadb引用了持久化存储
helm repo add bitnami https://charts.bitnami.com/bitnami
helm install wordpress bitnami/wordpress \
--namespace=wordpress \
--create-namespace \
--set global.storageClass=nfs-client \
--set wordpressUsername=admin \
--set wordpressPassword=password \
--set replicaCount=2 \
--set service.type=NodePort \
--set service.nodePorts.http=30808 \
--set persistence.enabled=true \
--set persistence.size=10Gi \
--set volumePermissions.enabled=true \
--set mariadb.enabled=true \
--set mariadb.architecture=standalone \
--set mariadb.auth.rootPassword=password \
--set mariadb.auth.password=password \
--set mariadb.primary.persistence.enabled=true \
--set mariadb.primary.persistence.size=8Gi \
--set memcached.enabled=true \
--set wordpressConfigureCache=true
查看创建的pod及service:
[root@master ~]# kubectl -n wordpress get pods
NAME READY STATUS RESTARTS AGE
wordpress-798966ff57-8jfgb 1/1 Running 1 5m20s
wordpress-798966ff57-hk6gz 1/1 Running 0 5m20s
wordpress-mariadb-0 1/1 Running 0 5m20s
wordpress-memcached-67744fbd98-v8j8z 1/1 Running 0 5m20s
[root@master ~]# kubectl -n wordpress get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
wordpress ClusterIP 10.233.63.20 <none> 80/TCP,443/TCP 3m53s
wordpress-mariadb ClusterIP 10.233.21.68 <none> 3306/TCP 3m53s
wordpress-memcached ClusterIP 10.233.4.89 <none> 11211/TCP 3m53s
查看自动创建的pvc及pv
[root@master ~]# kubectl -n wordpress get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
data-wordpress-mariadb-0 Bound pvc-a677fa75-fdb0-4a3c-aa7e-de0e9be3baac 8Gi RWO nfs-client 141m
wordpress Bound pvc-c5b58ca0-73e3-4ef9-9113-9344524776c2 10Gi RWO nfs-client 6m40s
[root@master ~]# kubectl -n wordpress get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-a677fa75-fdb0-4a3c-aa7e-de0e9be3baac 8Gi RWO Delete Bound wordpress/data-wordpress-mariadb-0 nfs-client 141m
pvc-c5b58ca0-73e3-4ef9-9113-9344524776c2 10Gi RWO Delete Bound wordpress/wordpress nfs-client 6m42s
连接到NFS server查看本地数据
[root@nfs-server ~]# ll /nfs-share
total 24
drwxrwxrwx 3 root root 4096 Aug 16 14:23 wordpress-data-wordpress-mariadb-0-pvc-a54d15ff-fec7-4b0a-9c60-3181926409a7
drwxrwxrwx 3 root root 4096 Aug 16 13:58 wordpress-wordpress-pvc-cb951124-aec5-4650-8f1d-2a4850f5d34d
浏览器访问wordpress应用
示例2:创建一个tomcat应用进行验证
helm repo add bitnami https://charts.bitnami.com/bitnami
helm install tomcat bitnami/tomcat \
--namespace=apps \
--create-namespace \
--set global.storageClass=nfs-client \
--set replicaCount=2 \
--set service.type=NodePort \
--set service.nodePort=30809
查看创建的相关资源:
[root@master ~]# kubectl -n apps get pods
NAME READY STATUS RESTARTS AGE
tomcat-776b4846fd-7m8p2 1/1 Running 0 44m
tomcat-776b4846fd-w4vms 1/1 Running 0 2m54s
[root@master ~]# kubectl -n apps get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
tomcat NodePort 10.233.50.18 <none> 80:30809/TCP 44m
[root@master ~]# kubectl -n apps get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
tomcat Bound pvc-6c26f444-92f0-45bc-9884-b3ff87bbb338 8Gi RWO nfs-client 44m
[root@master ~]# kubectl -n apps get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-6c26f444-92f0-45bc-9884-b3ff87bbb338 8Gi RWO Delete Bound apps/tomcat nfs-client 44m
pvc-a54d15ff-fec7-4b0a-9c60-3181926409a7 8Gi RWO Delete Bound wordpress/data-wordpress-mariadb-0 nfs-client 53m
pvc-cb951124-aec5-4650-8f1d-2a4850f5d34d 10Gi RWO Delete Bound wordpress/wordpress nfs-client 53m
浏览器访问tomcat: