本文将介绍使用nfs-client-provisioner,把NFS Server给Kubernetes作为持久存储的后端,并且动态提供PV。
nfs-client-provisioner 是一个Kubernetes的简易NFS的外部provisioner,本身不提供NFS,需要现有的NFS服务器提供存储。
- PV以
${namespace}-${pvcName}-${pvName}
的命名格式提供(在NFS服务器上) - PV回收的时候以
archieved-${namespace}-${pvName}
的命名格式(在NFS服务器上)
确认nfs server 以及存在(此处不进行nfs的配置,具体可以参考网络资料)
一、配置rbac
nfs provision使用单独的serviceaccount账户,yaml内容如下:
apiVersion: v1
kind: ServiceAccount
metadata:
name: nfs-client-provisioner
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: nfs-client-provisioner ###我创建的权限给了最大。
namespace: default
rules:
- apiGroups:
- '*'
resources:
- '*'
verbs:
- get
- list
- watch
- create
- patch
- update
- exec
- delete
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: nfs-client-provisioner
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: nfs-client-provisioner
subjects:
- kind: ServiceAccount
name: nfs-client-provisioner
namespace: default
[root@master ~]# kubectl get ClusterRoleBinding | grep nfs
nfs-client-provisioner ClusterRole/nfs-client-provisioner 4d6h
[root@master ~]# kubectl get ClusterRole | grep nfs
nfs-client-provisioner 2023-03-23T07:40:14Z
[root@master ~]# kubectl get sa | grep nfs
nfs-client-provisioner 0 4d6h
[root@master ~]#
二、启动nfs provisioner
注意使用的镜像必须为最新,否则为出现(unexpected error getting claim reference: selfLink was empty, can’t make reference这个错误,之前网上查阅资料在api里面开启--feature-gates=RemoveSelfLink=false
这个参数,但是在1.20版本之后此参数已经弃用,1.24版本之后添加此参数会造成api无法启动问题。)
kind: Deployment
apiVersion: apps/v1
metadata:
name: nfs-client-provisioner
spec:
replicas: 1
selector:
matchLabels:
app: nfs-client-provisioner
strategy:
type: Recreate
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
serviceAccountName: nfs-client-provisioner ###上面创建的sa的名称
containers:
- name: nfs-client-provisioner
image: registry.cn-beijing.aliyuncs.com/mydlq/nfs-subdir-external-provisioner:v4.0.0 ####注意使用tag 为latest的镜像会出现上述的问题。
imagePullPolicy: IfNotPresent
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: fuseim.pri/ifs ###provisioner 名称
- name: NFS_SERVER ###nfs的server地址
value: 192.168.5.240
- name: NFS_PATH ###nfs挂载的路径
value: /data
volumes:
- name: nfs-client-root
nfs:
server: 192.168.5.240 ###同上
path: /data
[root@master ~]#
[root@master ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-6fb6f65f8b-4sxgp 1/1 Running 0 4d6h
三、创建storageclass
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: managed-nfs-storage
provisioner: fuseim.pri/ifs ####上述定义的provisioner 名称
[root@master ~]# kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
managed-nfs-storage fuseim.pri/ifs Delete Immediate false 4d6h
四、创建pvc测试
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: test-claim
spec:
storageClassName: managed-nfs-storage ###sc name
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi ###pvc大小
[root@master ~]# kubectl get pvc | grep test
test-claim Bound pvc-82480978-dff0-4b82-bd15-dcc14d0617de 1Gi RWX managed-nfs-storage 48s
[root@master ~]#
[root@master ~]#
[root@master ~]# kubectl get pv | grep pvc-82480978-dff0-4b82-bd15-dcc14d0617de
pvc-82480978-dff0-4b82-bd15-dcc14d0617de 1Gi RWX Delete Bound default/test-claim managed-nfs-storage 67s
五、删除pvc
[root@master ~]# kubectl delete pvc test-claim
persistentvolumeclaim "test-claim" deleted
[root@master ~]#
[root@master ~]# kubectl get pvc | grep test
[root@master ~]# kubectl get pv | grep pvc-82480978-dff0-4b82-bd15-dcc14d0617de
查看nfs 路径下的pvc,如下:
[root@master data]# ll | grep pvc-82480978-dff0-4b82-bd15-dcc14d0617de
drwxrwxrwx 2 root root 4096 Mar 27 22:30 archived-pvc-82480978-dff0-4b82-bd15-dcc14d0617de