kubernetes（k8s）PVC

概念

PVC 的全称是：PersistentVolumeClaim（持久化卷声明），PVC 是用户存储的一种声明，PVC 和 Pod 比较类似，Pod 消耗的是节点，PVC 消耗的是 PV 资源，Pod 可以请求 CPU 和内存，而 PVC 可以请求特定的存储空间和访问模式。对于真正使用存储的用户不需要关心底层的存储实现细节，只需要直接使用 PVC
即可。

环境描述

本文中有一个master，3个node节点
192.168.8.81 k8master1.meng.com k8master1 k8kubapi.meng.com k8kubapi
192.168.8.84 k8node1.meng.com k8node1
192.168.8.85 k8node2.meng.com k8node2
192.168.8.86 k8node3.meng.com k8node3

准备工作

查看master和node节点的状态

[root@k8master1 ~]# kubectl get nodes
NAME                 STATUS   ROLES           AGE   VERSION
k8master1.meng.com   Ready    control-plane   22d   v1.28.1
k8node1.meng.com     Ready    <none>          22d   v1.28.1
k8node2.meng.com     Ready    <none>          22d   v1.28.1
k8node3.meng.com     Ready    <none>          22d   v1.28.1

需要在所有节点安装 nfs 客户端程序，必须在所有节点都安装 nfs 客户端，否则可能会导致 PV 挂载不上的问题。
安装命令如下：

yum -y install  nfs-utils rpcbind
systemctl start rpcbind.service 
systemctl start nfs.service 

在master节点创建一些文件夹，更改属主

mkdir -p /nfs1/data/01 /nfs1/data/02 /nfs1/data/03
chown -R nfsnobody.nfsnobody /nfs1/  

将新建的文件夹共享

[root@k8master1 ~]# vim /etc/exports

添加如下内容

/nfs1/data/01  192.168.8.0/24(rw,sync,root_squash,all_squash)
/nfs1/data/02  192.168.8.0/24(rw,sync,root_squash,all_squash)
/nfs1/data/03  192.168.8.0/24(rw,sync,root_squash,all_squash)

从node节点尝试访问master节点共享的NFS文件夹，所有的节点都需要测试一下

[root@k8node1 containerd]# showmount -e 192.168.8.81
Export list for 192.168.8.81:
/nfs1/data/03 192.168.8.0/24
/nfs1/data/02 192.168.8.0/24
/nfs1/data/01 192.168.8.0/24
[root@k8node1 containerd]# mount -t nfs 192.168.8.81:/nfs1/data/01 /mnt
[root@k8node1 /]# umount /mnt

部署PV

创建了3个PV，大小为10M，1G，3G

[root@k8master1 ~]# cat pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv01-10m
spec:
  capacity:
    storage: 10M
  accessModes:
    - ReadWriteMany
  storageClassName: nfs
  nfs:
    path: /nfs1/data/01
    server: 192.168.8.81
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv02-1gi
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteMany
  storageClassName: nfs
  nfs:
    path: /nfs1/data/02
    server: 192.168.8.81
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv03-3gi
spec:
  capacity:
    storage: 3Gi
  accessModes:
    - ReadWriteMany
  storageClassName: nfs
  nfs:
    path: /nfs1/data/03
    server: 192.168.8.81

创建PV

[root@k8master1 ~]# kubectl apply -f pv.yaml

查看PV状态

[root@k8master1 ~]# kubectl get pv
NAME       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE
pv01-10m   10M        RWX            Retain           Available           nfs                     4s
pv02-1gi   1Gi        RWX            Retain           Available           nfs                     4s
pv03-3gi   3Gi        RWX            Retain           Available           nfs                     4s 

创建PVC

[root@k8master1 ~]# cat pvc.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: nginx-pvc
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 200Mi
  storageClassName: nfs

运维YAML文件创建PVC

[root@k8master1 ~]# kubectl apply -f pvc.yaml

查看PV，已有一个PV处于绑定状态

[root@k8master1 ~]# kubectl get pv
NAME       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM               STORAGECLASS   REASON   AGE
pv01-10m   10M        RWX            Retain           Available                       nfs                     42h
pv02-1gi   1Gi        RWX            Retain           Bound       default/nginx-pvc   nfs                     42h
pv03-3gi   3Gi        RWX            Retain           Available                       nfs                     42h

发现这个pv02-1gi绑定了我们刚才的pvc，为什么是这个，因为我们申请的是200M，01太小，03太大，k8s帮我们自动做了选择

创建Pod绑定PVC

[root@k8master1 ~]# cat pvc-deploy2.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx-deploy-pvc
  name: nginx-deploy-pvc
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx-deploy-pvc
  template:
    metadata:
      labels:
        app: nginx-deploy-pvc
    spec:
      containers:
      - image: nginx
        name: nginx
        volumeMounts:
        - name: html
          mountPath: /usr/share/nginx/html
      volumes:
        - name: html
          persistentVolumeClaim:
            claimName: nginx-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-pvc
  labels:
    app: nginx-pvc
spec:
  type: NodePort
  ports:
  - port: 80
    targetPort: web              #容器端口或名字
  selector:
    app: nginx-pvc
[root@k8master1 ~]#

执行文件创建pod，创建deployment和service，并暴露给node

[root@k8master1 ~]# kubectl apply -f pvc-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx-deploy-pvc
  name: nginx-deploy-pvc
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx-deploy-pvc
  template:
    metadata:
      labels:
        app: nginx-deploy-pvc
    spec:
      containers:
      - image: nginx
        name: nginx
        ports:
        - containerPort: 80
          name: web
        volumeMounts:
        - name: www
          mountPath: /usr/share/nginx/html
      volumes:
        - name: www
          persistentVolumeClaim:
            claimName: nginx-pvc
---

apiVersion: v1
kind: Service
metadata:
  name: nginx-pvc
  labels:
    app: nginx-pvc
spec:
  type: NodePort
  ports:
    - port: 80
      targetPort: 80
      nodePort: 30004
  selector:
    app: nginx-deploy-pvc

我们进入到02里面新建个文件

[root@k8master1 ~]# cd /nfs1/data/02
[root@k8master1 ~]# echo 111 > index.html

交互访问docker，可以看到新创建的index文件

[root@k8master1 02]# kubectl exec -it nginx-deploy-pvc-7c466b8668-cfpsg  -- bash
root@nginx-deploy-pvc-7c466b8668-cfpsg:/# cd /usr/share/nginx/html/
root@nginx-deploy-pvc-7c466b8668-cfpsg:/usr/share/nginx/html# ls
index.html
root@nginx-deploy-pvc-7c466b8668-cfpsg:/usr/share/nginx/html#

创建的pod可以curl一下看看是否可以访问，如果不能访问可以参考K8S遇到的问题的文章。

[root@k8master1 ~]# kubectl get pods -o wide
NAME                                READY   STATUS    RESTARTS        AGE   IP            NODE               NOMINATED NODE   READINESS GATES
demoapp-7c58cd6bb-7qlrd             1/1     Running   1 (16m ago)     22d   10.244.1.15   k8node1.meng.com   <none>           <none>
demoapp-7c58cd6bb-f84kp             1/1     Running   1 (4m50s ago)   22d   10.244.3.12   k8node3.meng.com   <none>           <none>
demoapp-7c58cd6bb-ldrzf             1/1     Running   1 (8m50s ago)   22d   10.244.2.13   k8node2.meng.com   <none>           <none>
mypod                               1/1     Running   1 (8m50s ago)   21d   10.244.2.12   k8node2.meng.com   <none>           <none>
nginx-deploy-pvc-64b6b6bb47-csrnr   1/1     Running   1 (4m50s ago)   20m   10.244.3.11   k8node3.meng.com   <none>           <none>
nginx-deploy-pvc-64b6b6bb47-wcsbc   1/1     Running   1 (16m ago)     20m   10.244.1.16   k8node1.meng.com   <none>           <none>
[root@k8master1 ~]# curl 10.244.1.16
111
[root@k8master1 ~]#

通过浏览器可以访问暴露的端口来访问Nginx了，master节点和node节点都可以

手动创建deployment和service

kubectl create deploy nginx-deploy-pvc --image=nginx:latest --port=80 --replicas=2
kubectl expose deploy nginx-deploy-pvc --name=nginx-pvc --type=NodePort --port=80 --target-port=80