KubeSphere启用可插拔组件失败

在安装 KubeSphere web管理页面中安装应用商店失败，查看日志：

kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f

失败信息如下：

TASK [common : debug] **********************************************************
ok: [localhost] => {
    "msg": [
        "1. check the storage configuration and storage server",
        "2. make sure the DNS address in /etc/resolv.conf is available",
        "3. execute 'kubectl logs -n kubesphere-system -l job-name=minio-make-bucket-job' to watch logs",
        "4. execute 'helm -n kubesphere-system uninstall ks-minio && kubectl -n kubesphere-system delete job minio-make-bucket-job'",
        "5. Restart the installer pod in kubesphere-system namespace"
    ]
}

TASK [common : fail] ***********************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "It is suggested to refer to the above methods for troubleshooting problems ."}
localhost                  : ok=40   changed=28   unreachable=0    failed=1    skipped=81   rescued=0    ignored=0   


PLAY RECAP *********************************************************************
localhost                  : ok=40   changed=28   unreachable=0    failed=1    skipped=81   rescued=0    ignored=0

可以看到第一条信息：1. check the storage configuration and storage server（检查存储配置和存储服务器）

查看默认存储类的名称：

kubectl get storageclass

出现了两条默认的 storageclass

NAME                    PROVISIONER                                   RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
local (default)         openebs.io/local                              Delete          WaitForFirstConsumer   false                  31h
nfs-storage (default)   k8s-sigs.io/nfs-subdir-external-provisioner   Delete          Immediate              false                  22h

第一个 local 是在安装 KubeSphere 时由于没有安装默认存储类型（nfs-storage）， KubeSphere 自动安装的默认存储类型。

在 kubesphere 文档中心中(https://v1-0.docs.kubesphere.io/docs/zh-CN/installation/storage-configuration/)看到一句话：

说明： 集群中不可同时存在两个默认存储类型，若要指定默认存储类型前请先确保当前集群中无默认存储类型。

于是删除第一个默认安装的存储类型：

1. 查看默认存储类的名称：

   kubectl get storageclass
   

   默认存储类通常是名称为 standard 或 default 的存储类。

2. 查看使用默认存储类的持久卷：

   kubectl get pv --sort-by=.spec.capacity.storage
   

   这将列出所有持久卷，并按存储容量排序。查找使用默认存储类的持久卷并记录它们的名称。

3. 将默认存储类的注释中的 is-default-class 标记设置为 false：

   kubectl patch storageclass <default-storage-class-name> -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
   

   其中 <default-storage-class-name> 是默认存储类的名称。

4. 将使用默认存储类的持久卷重新绑定到其他存储类上：

   kubectl patch pv <pv-name> -p '{"spec":{"storageClassName":"<new-storage-class-name>"}}'
   

   其中 <pv-name> 是持久卷的名称，<new-storage-class-name> 是您想要将持久卷重新绑定到的新存储类的名称。

5. 确认所有持久卷已经成功绑定到新的存储类上：

   kubectl get pv --sort-by=.spec.capacity.storage
   

   确认所有使用默认存储类的持久卷都已经重新绑定到新的存储类上后，您可以使用 kubectl delete storageclass <default-storage-class-name> 命令来删除默认存储类。

6. 删除默认存储类：

   kubectl delete storageclass <default-storage-class-name>
   

以下是我自己操作的命令：

[root@master ~]# kubectl get storageclass
NAME                    PROVISIONER                                   RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
local (default)         openebs.io/local                              Delete          WaitForFirstConsumer   false                  31h
nfs-storage (default)   k8s-sigs.io/nfs-subdir-external-provisioner   Delete          Immediate              false                  22h

[root@master ~]# kubectl get pv --sort-by=.spec.capacity.storage
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                                             STORAGECLASS   REASON   AGE
pvc-101ab0ee-ffe9-4f6b-8ec5-4d4d7c9e9cbf   2Gi        RWO            Delete           Bound    his/redis-pvc-his-redis-1                                         nfs-storage             133m
pvc-1207a53a-e20e-4ddc-a97a-a69f8ac81b97   2Gi        RWO            Delete           Bound    his/redis-pvc-his-redis-2                                         nfs-storage             133m
pvc-91eb95eb-f6db-45e7-9978-e9f8cc386ad3   2Gi        RWO            Delete           Bound    his/redis-pvc-his-redis-0                                         nfs-storage             145m
pvc-02a216d8-3200-416a-b601-c4ecd2d26c27   5Gi        RWO            Delete           Bound    his/mysql-pvc                                                     nfs-storage             22h
pvc-4ce327e8-d647-41ad-8d55-77f41598d5b8   20Gi       RWO            Delete           Bound    kubesphere-monitoring-system/prometheus-k8s-db-prometheus-k8s-0   local                   30h
pvc-d3669cba-1ed0-482a-bdb3-65b63054efbd   20Gi       RWO            Delete           Bound    kubesphere-monitoring-system/prometheus-k8s-db-prometheus-k8s-1   local                   30h
# 上面可以看到有两个绑定在 local 的持久卷 pvc-4ce327e8-d647-41ad-8d55-77f41598d5b8 和 pvc-d3669cba-1ed0-482a-bdb3-65b63054efbd

[root@master ~]# kubectl patch storageclass local -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
storageclass.storage.k8s.io/local patched

[root@master ~]# kubectl get storageclass
NAME                    PROVISIONER                                   RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
local (default)         openebs.io/local                              Delete          WaitForFirstConsumer   false                  31h
nfs-storage (default)   k8s-sigs.io/nfs-subdir-external-provisioner   Delete          Immediate              false                  23h


[root@master ~]# kubectl patch pv pvc-4ce327e8-d647-41ad-8d55-77f41598d5b8 -p '{"spec":{"storageClassName":"nfs-storage"}}'
persistentvolume/pvc-4ce327e8-d647-41ad-8d55-77f41598d5b8 patched


[root@master ~]# kubectl patch pv pvc-d3669cba-1ed0-482a-bdb3-65b63054efbd -p '{"spec":{"storageClassName":"nfs-storage"}}'
persistentvolume/pvc-d3669cba-1ed0-482a-bdb3-65b63054efbd patched


[root@master ~]# kubectl get storageclass
NAME                    PROVISIONER                                   RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
local (default)         openebs.io/local                              Delete          WaitForFirstConsumer   false                  31h
nfs-storage (default)   k8s-sigs.io/nfs-subdir-external-provisioner   Delete          Immediate              false                  23h


[root@master ~]# kubectl get pv --sort-by=.spec.capacity.storage
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                                             STORAGECLASS   REASON   AGE
pvc-101ab0ee-ffe9-4f6b-8ec5-4d4d7c9e9cbf   2Gi        RWO            Delete           Bound    his/redis-pvc-his-redis-1                                         nfs-storage             146m
pvc-1207a53a-e20e-4ddc-a97a-a69f8ac81b97   2Gi        RWO            Delete           Bound    his/redis-pvc-his-redis-2                                         nfs-storage             146m
pvc-91eb95eb-f6db-45e7-9978-e9f8cc386ad3   2Gi        RWO            Delete           Bound    his/redis-pvc-his-redis-0                                         nfs-storage             158m
pvc-02a216d8-3200-416a-b601-c4ecd2d26c27   5Gi        RWO            Delete           Bound    his/mysql-pvc                                                     nfs-storage             22h
pvc-4ce327e8-d647-41ad-8d55-77f41598d5b8   20Gi       RWO            Delete           Bound    kubesphere-monitoring-system/prometheus-k8s-db-prometheus-k8s-0   nfs-storage             31h
pvc-d3669cba-1ed0-482a-bdb3-65b63054efbd   20Gi       RWO            Delete           Bound    kubesphere-monitoring-system/prometheus-k8s-db-prometheus-k8s-1   nfs-storage             31h
# 此时 后面两个持久卷已经绑定在了 nfs-storage 存储类上


[root@master ~]# kubectl patch storageclass local -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
storageclass.storage.k8s.io/local patched (no change)

[root@master ~]# kubectl delete storageclass local
storageclass.storage.k8s.io "local" deleted

[root@master ~]# kubectl get storageclass

NAME                    PROVISIONER                                   RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
nfs-storage (default)   k8s-sigs.io/nfs-subdir-external-provisioner   Delete          Immediate           false                  23h

# 删除后，只剩一个默认存储类

最后重新启动 KubeSphere 应用商店安装成功：

#####################################################
###              Welcome to KubeSphere!           ###
#####################################################

Console: http://192.168.0.32:30881
Account: admin
Password: P@88w0rd

NOTES：
  1. After you log into the console, please check the
     monitoring status of service components in
     "Cluster Management". If any service is not
     ready, please wait patiently until all components 
     are up and running.
  2. Please change the default password after login.

#####################################################
https://kubesphere.io             2023-04-26 17:14:57
#####################################################