目录
方式一:nfs挂载到宿主机目录
方法二:在机器上搭建nfs,其余使用就可以手动静态创建所要的pv和pvc,或创建pvc动态创建pv
修改kube-apiserver.yaml ,增加 - --feature-gates=RemoveSelfLink=false
创建RBAC授权
两种方式:
方式一:nfs挂载到宿主机目录
优点: 简单易用,无需额外支持 缺点:依赖宿主机磁盘容量,pod与宿主机存在强耦合,不利于管理。需要指定标签,当pod部署多个副本并分配到不同host时,数据不共享;当要驱赶pod时,数据不同步;当node故障时,数据易丢失;
1.node打标签
kubectl label nodes master25 monitor=nfsserver
2.all.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nfs-busybox
spec:
replicas: 2
template:
metadata:
labels:
name: nfs-busybox
spec:
containers:
- name: busybox
image: busybox
command:
- sh
- -c
- 'while true; do date > /mnt/index.html; hostname >> /mnt/index.html; sleep $(($RANDOM % 5 + 5)); done'
imagePullPolicy: IfNotPresent
volumeMounts:
# name must match the volume name below
- name: nfs
mountPath: "/mnt"
volumes:
- name: nfs
persistentVolumeClaim:
claimName: nfs
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: nfs
spec:
accessModes:
- ReadWriteMany
storageClassName: ""
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteMany
nfs:
# FIXME: use the right IP
# get the server ip from the kube service
server: 10.107.89.35
#server: nfs-server.default.svc.cluster.local
path: "/nfsdemo/"
---
apiVersion: v1
kind: Service
metadata:
name: nfs-server
spec:
ports:
- name: nfs
port: 2049
targetPort: 2049
protocol: TCP
- name: mountd
port: 20048
- name: rpcbind
port: 111
selector:
name: nfs-server
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nfs-server
namespace: default
spec:
replicas: 1
template:
metadata:
labels:
name: nfs-server
spec:
nodeSelector:
label: nfsserver
containers:
- name: nfs-server
image: googlecontainer/volume-nfs:0.8
ports:
- name: nfs
containerPort: 2049
- name: mountd
containerPort: 20048
- name: rpcbind
containerPort: 111
securityContext:
privileged: true
volumeMounts:
- mountPath: /exports
name: nfs-folder
volumes:
- name: nfs-folder
hostPath:
path: "/home/dkongjian/nfs-share"
方法二:在机器上搭建nfs,其余使用就可以手动静态创建所要的pv和pvc,或创建pvc动态创建pv
优点:可做高可用配置,预防单点故障
缺点:切记一定要配置成开机自启动,
架构是:搭建StorageClass+NFS
1
.创建一个可用的NFS Serve
2
.创建Service Account.这是用来管控NFS provisioner在k8s集群中运行的权限
3
.创建StorageClass.负责建立PVC并调用NFS provisioner进行预定的工作,并让PV与PVC建立管理
4
.创建NFS provisioner.有两个功能,一个是在NFS共享目录下创建挂载点(volume),另一个则是建了PV并将PV与NFS的挂载点建立关联
1.安装nfs-server
sudo apt install nfs-kernel-server
其余node安装
apt-get install nfs-common
2.创建文件系统
sudo mkdir -p /data/nfsboot/
sudo chmod
777
/data/nfsboot/
3.配置文件系统
vim /etc/exports
/data/nfsboot/ *(rw,sync,no_root_squash)
重启服务
sudo service nfs-kernel-server restart
service nfs-kernel-server enable
showmount测试
root
@master24
:/data/nfsboot# showmount -e
127.0
.
0.1
Export list
for
127.0
.
0.1
:
/data/nfsboot *
修改kube-apiserver.yaml ,增加 - --feature-gates=RemoveSelfLink=false
原因是:
创建pvc后状态一直是pending,因为原来是1.20版本(我的是1.22.4)默认禁止使用selfLink。
创建RBAC授权
rbac.yaml: #唯一需要修改的地方只有namespace,根据实际情况定义
wget https:
//raw.githubusercontent.com/kubernetes-incubator/external-storage/master/nfs-client/deploy/rbac.yaml
# kubectl apply -f rbac.yaml
3.创建NFS资源的StorageClass
class.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: managed-nfs-storage
provisioner: fuseim.pri/ifs # or choose another name, must match deployment
's env PROVISIONER_NAME'
parameters:
archiveOnDelete:
"false"
4.创建nfs-client-provisioner自动配置程序,以便自动创建持久卷(PV)
自动创建的 PV 以 ${namespace}-${pvcName}-${pvName} 的命名格式创建在 NFS 上 当这个 PV 被回收后会以 archieved-${namespace}-${pvcName}-${pvName} 的命名格式存在 NFS 服务器上
deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nfs-client-provisioner
labels:
app: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace:
default
spec:
replicas:
1
strategy:
type: Recreate
selector:
matchLabels:
app: nfs-client-provisioner
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
serviceAccountName: nfs-client-provisioner
containers:
- name: nfs-client-provisioner
image: quay.io/external_storage/nfs-client-provisioner:latest
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: fuseim.pri/ifs
- name: NFS_SERVER
value:
192.168
.
50.24
- name: NFS_PATH
value: /data/nfsboot
volumes:
- name: nfs-client-root
nfs:
server:
192.168
.
50.24
path: /data/nfsboot
5.测试
# cat statefulset-nfs.yaml
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
ports:
- port:
80
name: web
clusterIP: None
selector:
app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: nfs-web
spec:
serviceName:
"nginx"
replicas:
3
selector:
matchLabels:
app: nfs-web # has to match .spec.template.metadata.labels
template:
metadata:
labels:
app: nfs-web
spec:
terminationGracePeriodSeconds:
10
containers:
- name: nginx
image: nginx:
1.7
.
9
ports:
- containerPort:
80
name: web
volumeMounts:
- name: www
mountPath: /usr/share/nginx/html
volumeClaimTemplates:
- metadata:
name: www
annotations:
volume.beta.kubernetes.io/storage-
class
: managed-nfs-storage
spec:
accessModes: [
"ReadWriteOnce"
]
resources:
requests:
storage: 1Gi
查看 Pod/PV/PVC
root
@master24
:/opt/yaml/nfs# kubectl get pods
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-5cd65b7586-tbb7l
1
/
1
Running
0
24m
nfs-web-
0
1
/
1
Running
0
10m
nfs-web-
1
1
/
1
Running
0
9m41s
nfs-web-
2
1
/
1
Running
0
8m49s
root
@master24
:/opt/yaml/nfs# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-c67257ab-
3694
-47b5-afe1-d1cf5ebe098d 1Gi RWO Delete Bound
default
/www-nfs-web-
1
managed-nfs-storage 9m49s
pvc-e9ecc66a-f4db-4bfb-a66f-56a13a969e1a 1Gi RWO Delete Bound
default
/www-nfs-web-
0
managed-nfs-storage 11m
pvc-ea6fb9d2-c621-4d03-97f7-d54c570a7e22 1Gi RWO Delete Bound
default
/www-nfs-web-
2
managed-nfs-storage 8m56s
root
@master24
:/opt/yaml/nfs# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
www-nfs-web-
0
Bound pvc-e9ecc66a-f4db-4bfb-a66f-56a13a969e1a 1Gi RWO managed-nfs-storage 16m
www-nfs-web-
1
Bound pvc-c67257ab-
3694
-47b5-afe1-d1cf5ebe098d 1Gi RWO managed-nfs-storage 9m53s
www-nfs-web-
2
Bound pvc-ea6fb9d2-c621-4d03-97f7-d54c570a7e22 1Gi RWO managed-nfs-storage 9m1s
查看 nfs server 目录中信息,同时各子目录中内容为空
root
@master24
:/opt/yaml/nfs# ls -l /data/nfsboot/
total
20
drwxrwxrwx
2
root root
4096
Nov
29
13
:
59
default
-www-nfs-web-
0
-pvc-e9ecc66a-f4db-4bfb-a66f-56a13a969e1a
drwxrwxrwx
2
root root
4096
Nov
29
14
:
01
default
-www-nfs-web-
1
-pvc-c67257ab-
3694
-47b5-afe1-d1cf5ebe098d
drwxrwxrwx
2
root root
4096
Nov
29
14
:
02
default
-www-nfs-web-
2
-pvc-ea6fb9d2-c621-4d03-97f7-d54c570a7e22
drwxr-xr-x
2
root root
4096
Nov
27
15
:
08
test
-rw-r--r--
1
root root
6
Nov
26
17
:
47
test.txt