unexpected error getting claim reference: selfLink was empty, can‘t make reference

weixin_888988

已于 2022-10-09 21:03:01 修改

阅读量1.9k

点赞数

分类专栏： k8s 文章标签： kubernetes docker 容器

于 2022-10-09 20:40:14 首次发布

原文链接：https://blog.csdn.net/w2909526/article/details/117912323

版权

k8s 专栏收录该内容

19 篇文章 0 订阅

订阅专栏

问题：

在使用nfs创建storageclass 实现存储的动态加载
分别创建 rbac、nfs-deployment、nfs-storageclass之后都正常运行
但在创建pvc时一直处于pending状态
kubectl describe pvc test-claim 查看pvc信息提示如下

Name:          test-claim
Namespace:     default
StorageClass:  managed-nfs-storage
Status:        Pending
Volume:        
Labels:        <none>
Annotations:   volume.beta.kubernetes.io/storage-class: managed-nfs-storage
               volume.beta.kubernetes.io/storage-provisioner: fuseim.pri/ifs
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      
Access Modes:  
VolumeMode:    Filesystem
Used By:       <none>
Events:
  Type    Reason                Age               From                         Message
  ----    ------                ----              ----                         -------
  Normal  ExternalProvisioning  2s (x4 over 19s)  persistentvolume-controller  waiting for a volume to be created, either by external provisioner "fuseim.pri/ifs" or manually created by system administrator

查找无果然后查看nfs-pod状态，报错如下

[root@master manifests]# kubectl logs -f nfs-client-provisioner-6cf7cdc4fd-c6tr7
I1009 10:53:12.696646       1 leaderelection.go:185] attempting to acquire leader lease  default/qgg-nfs-storage...
"default/postgressqldb3-pvc" class "managed-nfs-storage": started
I1009 11:53:30.182237       1 controller.go:987] provision "default/sklmapp-pvc" class "managed-nfs-storage": started
E1009 11:53:30.188308       1 controller.go:1004] provision "default/postgressqldb3-pvc" class "managed-nfs-storage": unexpected error getting claim reference: selfLink was empty, can't make reference
E1009 11:53:30.188914       1 controller.go:1004] provision "default/sklmapp-pvc" class "managed-nfs-storage": unexpected error getting claim reference: selfLink was empty, can't make reference
I1009 12:08:30.182002       1 controller.go:987] provision "default/postgressqldb3-pvc" class "managed-nfs-storage": started
I1009 12:08:30.182090       1 controller.go:987] provision "default/sklmapp-pvc" class "managed-nfs-storage": started
E1009 12:08:30.190163       1 controller.go:1004] provision "default/sklmapp-pvc" class "managed-nfs-storage": unexpected error getting claim reference: selfLink was empty, can't make reference
E1009 12:08:30.190196       1 controller.go:1004] provision "default/postgressqldb3-pvc" class "managed-nfs-storage": unexpected error getting claim reference: selfLink was empty, can't make reference
^C

selfLink was empty 在k8s集群 v1.20之前都存在，在v1.20之后被删除，需要在/etc/kubernetes/manifests/kube-apiserver.yaml 添加参数
增加 - --feature-gates=RemoveSelfLink=false

spec:
  containers:
  - command:
    - kube-apiserver
    - --feature-gates=RemoveSelfLink=false

添加之后使用kubeadm部署的集群会自动加载部署pod

Just a note about /etc/kubernetes/manifests/kube-apiserver.yaml. 
When running the control-plane as an in-cluster service you should not need to apply changes to this file ever (via kubectl apply). 
As soon as you save the change to disk the apiserver pods will 
detect it and should conduct a Rolling Update. 
You can watch the progress with watch kubectl get pods --namespace kube-system. 
You can confirm the behavior after deleting the provisioner pod and letting it restart.

kubeadm安装的apiserver是Static Pod，它的配置文件被修改后，立即生效。
Kubelet 会监听该文件的变化，当您修改了 /etc/kubenetes/manifest/kube-apiserver.yaml 文件之后，kubelet 将自动终止原有的 kube-apiserver-{nodename} 的 Pod，并自动创建一个使用了新配置参数的 Pod 作为替代。
如果您有多个 Kubernetes Master 节点，您需要在每一个 Master 节点上都修改该文件，并使各节点上的参数保持一致。

这里需注意如果api-server启动失败需重新在执行一遍

kubectl apply -f /etc/kubernetes/manifests/kube-apiserver.yaml

方法2：

使用新版本的nfs-subdir-external-provisioner

Public link:

GitHub - kubernetes-sigs/nfs-subdir-external-provisioner: Dynamic sub-dir volume provisioner on a remote NFS server.

GitHub - groundhog2k/nfs-subdir-external-provisioner: Dynamic sub-dir volume provisioner on a remote NFS server.

Docker Hub

So far there are 3 steps before applying helm:

if you are using private/proxy registry make sure to point to the new image in gcr instead of the old one in quay.
delete MANUALLY 2 resources : the storageclass & the deployment

3. Then run helm apply

这个问题已经在github上有详细介绍
Using Kubernetes v1.20.0, getting "unexpected error getting claim reference: selfLink was empty, can't make reference" · Issue #25 · kubernetes-sigs/nfs-subdir-external-provisioner · GitHub