场景描述
glusterfs本身可以借助heketi来供k8s使用,如下图
有个场景:
- glusterfs是先于k8s安装的,机器已经没有新的磁盘供你单独给Heketi使用了
- 你没有权限创建新的卷,你只能使用提供给你的卷
针对上面的场景,可以使用下面的架构
部署
- 服务器A和服务器B上,部署NFS服务
yum -y install nfs-utils rpcbind
systemctl start rpcbind
systemctl start nfs
- 服务器A和服务器B上,配置NFS服务
# /ai-data/nfs-data 是我服务器上的挂载点
# glusterfs的卷要打开 nfs的支持
/ai-data/nfs-data *(rw,fsid=0,no_root_squash)
exportfs -rv
# 执行完上面命令后如果能看到正常的输出,则证明正常
- 配置k8s service
yaml文件
apiVersion: v1
kind: Service
metadata:
name: nfs-server-svc
spec:
ports:
- name: tcp
port: 2049
protocol: TCP
- name: udp
port: 2049
protocol: UDP
---
kind: Endpoints
apiVersion: v1
metadata:
name: nfs-server-svc
namespace: default
subsets:
- addresses:
- ip: 192.168.0.213 # 服务器A
ports:
- name: tcp
port: 2049
protocol: TCP
- name: udp
port: 2049
protocol: UDP
- addresses:
- ip: 192.168.0.214 # 服务器B
ports:
- name: tcp
port: 2049
protocol: TCP
- name: udp
port: 2049
protocol: UDP
上面使用nodeport,如果没有需求的话可以不使用
- 部署nfs provisioner
apiVersion: apps/v1
kind: Deployment
metadata:
name: nfs-client-provisioner-1215
labels:
app: nfs-client-provisioner-new
# replace with namespace where provisioner is deployed
namespace: ha-test-env # 命名空间要注意
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: nfs-client-provisioner-1215
template:
metadata:
labels:
app: nfs-client-provisioner-1215
spec:
serviceAccountName: nfs-client-provisioner
containers:
- name: nfs-client-provisioner
securityContext:
privileged: true
image: quay.io/external_storage/nfs-client-provisioner:latest
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: fuseim.pri/ifsnew # 此处的名字要注意
- name: NFS_SERVER
value: 10.96.64.180 # 此处是cluster ip
- name: NFS_PATH
value: / # 此处要是根目录,因为是NFS4
volumes:
- name: nfs-client-root
nfs:
server: 10.96.64.180
path: / # 此处要是根目录,因为是NFS4
- 配置 storageclass
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: nfs-storage-1215
provisioner: fuseim.pri/ifsnew
parameters:
onDelete: delete
- 测试
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: magpie-claim
spec:
storageClassName: nfs-storage-1215
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 100Mi #请求50M的空间
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: test-pvc-deployment
labels:
app: mg-tornado
spec:
replicas: 1
selector:
matchLabels:
app: mg-tornado
template:
metadata:
labels:
app: mg-tornado
spec:
nodeSelector:
use: magpie
containers:
- name: busybox
image: busybox
volumeMounts:
- mountPath: /db
name: db
volumes:
- name: db
persistentVolumeClaim: #指定pvc,注意下面声明的pvc指向的是上面定义的pvc名称
claimName: magpie-claim
- 进入容器后,对容器内的挂载点进行数据的写入,如果可以写入则证明配置成功