ceph 创建 static pvc

文档

Static PVC with ceph-csi
secret.yaml
Filesystem Storage Overview

实操

按照Static PVC with ceph-csi 操作，中间一些注意的地方

1. 创建 secret
   内容直接 copy 的 kubectl -n root-ceph get secret rook-csi-cephfs-node，记得修改adminID -> userID 和adminKey -> userKey

apiVersion: v1
data:
  userID: Y3NpLWNlcGhmcy1ub2Rl
  userKey: QVFEZmwrbGtaSXpZSnhBQUV0b1NSNXhRSWdiYVJHbW4rUVNqQmc9PQ==
kind: Secret
metadata:
  name: rook-csi-cephfs-node
type: kubernetes.io/rook

2. 创建 subvolume
   例子直接 get 的输出是 root_path, 实际是全路径,

# ceph fs subvolume create myfs testSubVolume csi --size=1073741824
# ceph fs subvolume getpath myfs testSubVolume csi
/volumes/csi/testSubVolume/d241d1ae-6ba7-4042-b9dc-4f4840c80512

3. pv、pvc 和 pod

apiVersion: v1
kind: PersistentVolume
metadata:
  name: cephfs-static-pv
spec:
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 1Gi
  csi:
    driver: rook-ceph.cephfs.csi.ceph.com
    nodeStageSecretRef:
      name: rook-csi-cephfs-node
      namespace: default
    volumeAttributes:
      # Required options from storageclass parameters need to be added in volumeAttributes
      clusterID: rook-ceph
      fsName: myfs
      staticVolume: "true"
      mounter: kernel
      pool: myfs-data0
      # 记得这里把最后的路径去掉，否则报错 MountVolume.MountDevice failed for volume "cephfs-static-pv" : rpc error: code = Aborted desc = an operation with the given Volume ID cephfs-static-pv already exists
      #rootPath: /volumes/csi/testSubVolume/d241d1ae-6ba7-4042-b9dc-4f4840c80512
      rootPath: /volumes/csi/testSubVolume
    # volumeHandle can be anything, need not to be same
    # as PV name or volume name. keeping same for brevity
    volumeHandle: cephfs-static-pv
  persistentVolumeReclaimPolicy: Retain
  volumeMode: Filesystem
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: cephfs-static-pvc
  namespace: default
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  volumeMode: Filesystem
  # volumeName should be same as PV name
  volumeName: cephfs-static-pv
---
apiVersion: v1
kind: Pod
metadata:
  name: pod-csi-vol-test-static-1
spec:
  containers:
    - name: my-container
      image: nginx
      ports:
        - containerPort: 80
      volumeMounts:
        - name: my-volume
          mountPath: /usr/share/nginx/html
  volumes:
    - name: my-volume
      persistentVolumeClaim:
        claimName: cephfs-static-pvc

# kubectl get -f static_pvc.yaml 
NAME                          TYPE                 DATA   AGE
secret/rook-csi-cephfs-node   kubernetes.io/rook   4      21s

NAME                                CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                       STORAGECLASS   REASON   AGE
persistentvolume/cephfs-static-pv   1Gi        RWX            Retain           Bound    default/cephfs-static-pvc                           21s

NAME                                      STATUS   VOLUME             CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/cephfs-static-pvc   Bound    cephfs-static-pv   1Gi        RWX                           21s

NAME                            READY   STATUS     RESTARTS   AGE
pod/pod-csi-vol-test-static-1   0/1     Init:0/1   0          20s

# kubectl get pods
NAME                        READY   STATUS            RESTARTS   AGE
pod-csi-vol-test-static-1   0/1     PodInitializing   0          37s