kubekey初期尝试安装 KubeSphere单机和多机避坑指南

kubekey初期尝试安装 KubeSphere单机和多机避坑指南
准备工作
请注意开始前工作确定各个软件版本情况，本文章要想阅读比较舒服请还得有些Go开发经验

CentOS 7.9
KubeKey v1.21
KubeSphere v3.2.1
Docker 和 Kubernetes 根据支持进行选择：
获取支持可以通过如下命令：

开始前必须准备
请认真阅读对应官方的几个文档说明
https://kubesphere.io/zh/docs/v3.3/
根据自己的安装选择进行阅读，注意准备干净的机器环境

在开始进行安装前可以不安装Docker kk 都会帮解决

依赖说明，在刚安装好的 CentOS机器上会提示：socat、conntrack 必须条件要安装

 yum -y install socat conntrack-tools

关闭防火墙

systemctl stop firewalld
systemctl disable firewalld

单机安装

https://kubesphere.com.cn/docs/quick-start/all-in-one-on-linux/

单机安装一般没有什么坑，一般比较顺畅。

./kk create cluster  --with-kubesphere v3.2.1

多机安装

https://kubesphere.io/zh/docs/v3.3/installing-on-linux/introduction/multioverview/

./kk create cluster -f sample.yaml --with-kubesphere v3.2.1

遇到的坑
不部署 KubeSphere ，只是部署 Kubernetes
1.根据文档进行如下命令进行部署，多次发现 KubeSphere 一直没有部署成功。Docker ps 中进程也少

./kk create cluster -f sample.yaml

部署中提示到 Congratulations! Installation is successful. 进行技术

经过源码查证后发下如下：

查看 Congratulations! Installation is successful. 出现位置

mgr.Logger.Infoln("Congratulations! Installation is successful.")

网上看发现

if mgr.KsEnable && !noNetworkPlugin  
``
执行 else 才会进行打印

2.确认网络是否启动 noNetworkPlugin  
```golang
noNetworkPlugin := mgr.Cluster.Network.Plugin == "" || mgr.Cluster.Network.Plugin == "none"

查看代码发现和网络插件有关，已经配置啦

network:
    plugin: calico
    kubePodsCIDR: 10.233.64.0/18
    kubeServiceCIDR: 10.233.0.0/18

3.推出只有 mgr.KsEnable ，然后，本地进行代码调试发现。执行结果确实是 false 是不是有些坑，因此开始寻找 KsEnable 如何赋值

// pkg\util\executor\executor.go
mgr.KsEnable = executor.Cluster.KubeSphere.Enabled

// pkg\cluster\install\install.go
// Execute executes the tasks based on the parameters in the Manager.
func Execute(executor *executor.Executor) error {
    mgr, err := executor.CreateManager()
    if err != nil {
        return err
    }

    return ExecTasks(mgr)
}
// 找到出现位置啦 ksEnabled
// CreateCluster is used to create cluster based on the given parameters or configuration file.
func CreateCluster(clusterCfgFile, k8sVersion, ksVersion string, logger *log.Logger, ksEnabled, verbose, skipCheck, skipPullImages, inCluster, deployLocalStorage bool, downloadCmd, containerManager string) error 

// 找到啦，结果知道如果执行啦
// cmd 
clusterCmd.Flags().BoolVarP(&opt.Kubesphere, "with-kubesphere", "", false, "Deploy a specific version of kubesphere (default v3.2.0)")

最终执行命令行为

./kk create cluster -f sample.yaml --with-kubesphere v3.2.1

成功后容器运行情况

CONTAINER ID   IMAGE                              COMMAND                  CREATED          STATUS          PORTS     NAMES
baf675187bf3   kubesphere/ks-controller-manager   "controller-manager …"   22 minutes ago   Up 22 minutes             k8s_ks-controller-manager_ks-controller-manager-ffcb75d47-94m4m_kubesphere-system_02958091-2450-409d-9886-499c4e0cac51_0
67d7d965f9ae   kubesphere/ks-apiserver            "ks-apiserver --logt…"   22 minutes ago   Up 22 minutes             k8s_ks-apiserver_ks-apiserver-585c646769-t2svn_kubesphere-system_09069fe7-7935-4db9-ab91-64458a8ca171_0
761eaf9a302e   kubesphere/pause:3.4.1             "/pause"                 23 minutes ago   Up 23 minutes             k8s_POD_ks-controller-manager-ffcb75d47-94m4m_kubesphere-system_02958091-2450-409d-9886-499c4e0cac51_0
c6b32a2a73a4   kubesphere/pause:3.4.1             "/pause"                 23 minutes ago   Up 23 minutes             k8s_POD_ks-apiserver-585c646769-t2svn_kubesphere-system_09069fe7-7935-4db9-ab91-64458a8ca171_0
fdd577140de5   ad393d6a4d1b                       "/usr/local/bin/kube…"   24 minutes ago   Up 24 minutes             k8s_kube-rbac-proxy_node-exporter-87cjh_kubesphere-monitoring-system_b696d02a-67c2-4f66-ae23-611584effb9c_0
30d0cde82925   e5a616e4b9cf                       "/bin/node_exporter …"   24 minutes ago   Up 24 minutes             k8s_node-exporter_node-exporter-87cjh_kubesphere-monitoring-system_b696d02a-67c2-4f66-ae23-611584effb9c_0
14ebf2a30d75   kubesphere/pause:3.4.1             "/pause"                 24 minutes ago   Up 24 minutes             k8s_POD_node-exporter-87cjh_kubesphere-monitoring-system_b696d02a-67c2-4f66-ae23-611584effb9c_0
d8b95cd6fd08   kubesphere/ks-console              "docker-entrypoint.s…"   25 minutes ago   Up 25 minutes             k8s_ks-console_ks-console-65f4d44d88-zjwff_kubesphere-system_480ca96d-a59d-4e8e-9b09-a8102055fe35_0
b12287629525   kubesphere/pause:3.4.1             "/pause"                 25 minutes ago   Up 25 minutes             k8s_POD_ks-console-65f4d44d88-zjwff_kubesphere-system_480ca96d-a59d-4e8e-9b09-a8102055fe35_0
eeeb128cb4cc   296a6d5035e2                       "/coredns -conf /etc…"   28 minutes ago   Up 28 minutes             k8s_coredns_coredns-b5648d655-r6jrp_kube-system_ea0a3f0d-00a9-41eb-a7ad-1ea17f7b06fd_0
a1b34a125478   296a6d5035e2                       "/coredns -conf /etc…"   28 minutes ago   Up 28 minutes             k8s_coredns_coredns-b5648d655-x4vkb_kube-system_d58c2fd9-ea8c-4bcd-8e2c-a653c30a8085_0
bb1786f3a87b   76ba70f4748f                       "/usr/bin/kube-contr…"   28 minutes ago   Up 28 minutes             k8s_calico-kube-controllers_calico-kube-controllers-846b5f484d-8wkh2_kube-system_1bdc6739-1f3d-4c67-8250-e623368699eb_0
4b944ea67453   kubesphere/pause:3.4.1             "/pause"                 28 minutes ago   Up 28 minutes             k8s_POD_coredns-b5648d655-r6jrp_kube-system_ea0a3f0d-00a9-41eb-a7ad-1ea17f7b06fd_0
10db2405e267   kubesphere/pause:3.4.1             "/pause"                 28 minutes ago   Up 28 minutes             k8s_POD_coredns-b5648d655-x4vkb_kube-system_d58c2fd9-ea8c-4bcd-8e2c-a653c30a8085_0
7fac1afa011f   kubesphere/pause:3.4.1             "/pause"                 28 minutes ago   Up 28 minutes             k8s_POD_calico-kube-controllers-846b5f484d-8wkh2_kube-system_1bdc6739-1f3d-4c67-8250-e623368699eb_0
48672cdbfb5c   5ef66b403f4f                       "start_runit"            28 minutes ago   Up 28 minutes             k8s_calico-node_calico-node-hdpzh_kube-system_051df14a-34ae-499f-bbdd-07366619696d_0
1eb462971ffc   kubesphere/pause:3.4.1             "/pause"                 28 minutes ago   Up 28 minutes             k8s_POD_calico-node-hdpzh_kube-system_051df14a-34ae-499f-bbdd-07366619696d_0
ab2ce9c4b7ed   5340ba194ec9                       "/node-cache -locali…"   29 minutes ago   Up 29 minutes             k8s_node-cache_nodelocaldns-rg59j_kube-system_d086aca8-96e8-499a-9909-0ccf45d5226e_0
13313efe560b   kubesphere/pause:3.4.1             "/pause"                 29 minutes ago   Up 29 minutes             k8s_POD_nodelocaldns-rg59j_kube-system_d086aca8-96e8-499a-9909-0ccf45d5226e_0
9a572541e8f5   e08abd2be730                       "/usr/local/bin/kube…"   29 minutes ago   Up 29 minutes             k8s_kube-proxy_kube-proxy-mpnfm_kube-system_23816492-bc8f-4088-84c1-5d9efe5539b3_0
2e64c44a287d   kubesphere/pause:3.4.1             "/pause"                 29 minutes ago   Up 29 minutes             k8s_POD_kube-proxy-mpnfm_kube-system_23816492-bc8f-4088-84c1-5d9efe5539b3_0
21f49f777c1e   184ef4d127b4                       "kube-controller-man…"   29 minutes ago   Up 29 minutes             k8s_kube-controller-manager_kube-controller-manager-k8sm2_kube-system_d271ef4c070d96ebc6df16d3216d96d1_0
0369423e8d55   8e60ea3644d6                       "kube-scheduler --au…"   29 minutes ago   Up 29 minutes             k8s_kube-scheduler_kube-scheduler-k8sm2_kube-system_20b205cc89e8b92cd9c547d63399b9d1_0
eb0989c8c259   7b2ac941d4c3                       "kube-apiserver --ad…"   29 minutes ago   Up 29 minutes             k8s_kube-apiserver_kube-apiserver-k8sm2_kube-system_9a017f31440765b21daa62c26353fd94_0
af2dbfea5a02   kubesphere/pause:3.4.1             "/pause"                 30 minutes ago   Up 29 minutes             k8s_POD_kube-scheduler-k8sm2_kube-system_20b205cc89e8b92cd9c547d63399b9d1_0
84dbee62b4d7   kubesphere/pause:3.4.1             "/pause"                 30 minutes ago   Up 29 minutes             k8s_POD_kube-controller-manager-k8sm2_kube-system_d271ef4c070d96ebc6df16d3216d96d1_0
33c4423aafad   kubesphere/pause:3.4.1             "/pause"                 30 minutes ago   Up 29 minutes             k8s_POD_kube-apiserver-k8sm2_kube-system_9a017f31440765b21daa62c26353fd94_0