K8S集群搭建，以及可视化界面Dashboard，和在kubernetes上安装kubeSphere

目录

k8s集群环境搭建

1：版本要求

2：环境准备

3：k8s集群搭建

1、卸载以前的docker包，这个步骤正常新的linux不用执行

2、配置好yum源

3、安装docker,我们选择固定版本

4、安装K8s基础环境设置

5、安装kubelet、kubeadm、kubectl

1、下载需要的镜像

 2、给三个节点加上域名映射

6、初始化主节点

1、初始化

2、安装网络组件

3、node加入master节点

7、dashboard可视化界面搭建

1、下载dashboard.yaml文件

2、创建kubernetes-dashboard管理员角色

8、kubeSphere可视化界面安装

1、安装sc

2、安装metrics-server集群指标监控组件

3、安装KubeSphere

---

k8s集群环境搭建

博客写作时我在已经安装完成。陆陆续续用时三天，期间也踩过不少坑。部分安装过程无法完全贴出。

已经尽最大能力完善安装步骤。如果各位看官在安装时遇到问题，欢迎留言询问

关于虚拟机的准备，如果有不会设置的请留言，我可出虚拟机安装及静态ip设置教程

这是官方文档 ：什么是 KubeSphere

1：版本要求

Linux： CentOS7 64位

docker：20.10.7
kubelet：1.20.9 
kubeadm：1.20.9 
kubectl：1.20.9

2：环境准备

1：准备三台虚拟机分别作为master，node1，node2节点

给每台虚拟机设置静态ip，我这边设置的是  192.168.110.29，192.168.110.30，192.168.110.31

计划搭建三个节点 

192.168.110.29 ：k8s-master

192.168.110.30 ：k8s-node1

192.168.110.31 ：k8s-node2

内存按照官方的要求设置为：内存4GB,处理器2核，硬盘大小40G

3：k8s集群搭建

此处开始三个虚拟机都要进行设置

1、卸载以前的docker包，这个步骤正常新的linux不用执行

sudo yum remove docker \
                  docker-client \
                  docker-client-latest \
                  docker-common \
                  docker-latest \
                  docker-latest-logrotate \
                  docker-logrotate \
                  docker-engine

2、配置好yum源

sudo yum install -y yum-utils
sudo yum-config-manager \
--add-repo \
http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

3、安装docker,我们选择固定版本

yum install -y docker-ce-20.10.7 docker-ce-cli-20.10.7  containerd.io-1.4.6

可以使用最新版本，默认下载最新版本不需要指定版本号

sudo yum install -y docker-ce docker-ce-cli containerd.io

等待下载完成！

然后启动docker

systemctl enable docker --now

然后配置镜像加速

sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["https://970k7x82.mirror.aliyuncs.com"],
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker

这是三条命令，可以一起执行

4、安装K8s基础环境设置

每个虚拟机都需要执行操作，设置域名

hostnamectl set-hostname xxxx

我自己设置的为

hostnamectl set-hostname k8s-master
hostnamectl set-hostname k8s-node1 
hostnamectl set-hostname k8s-node2 

将 SELinux 设置为 permissive 模式（相当于将其禁用）

sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

关闭swap

swapoff -a  
sed -ri 's/.*swap.*/#&/' /etc/fstab

关闭防火墙

systemctl stop firewalld

允许 iptables 检查桥接流量

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

让上面的配置生效

sudo sysctl --system

5、安装kubelet、kubeadm、kubectl

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
   http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
exclude=kubelet kubeadm kubectl
EOF

sudo yum install -y kubelet-1.20.9 kubeadm-1.20.9 kubectl-1.20.9 --disableexcludes=kubernetes

sudo systemctl enable --now kubelet

1、下载需要的镜像

sudo tee ./images.sh <<-'EOF'
#!/bin/bash
images=(
kube-apiserver:v1.20.9
kube-proxy:v1.20.9
kube-controller-manager:v1.20.9
kube-scheduler:v1.20.9
coredns:1.7.0
etcd:3.4.13-0
pause:3.2
)
for imageName in ${images[@]} ; do
docker pull registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/$imageName
done
EOF

chmod +x ./images.sh && ./images.sh

 2、给三个节点加上域名映射

echo "192.168.110.29  k8s-master" >> /etc/hosts

echo "192.168.110.30  k8s-master" >> /etc/hosts

echo "192.168.110.31  k8s-master" >> /etc/hosts

6、初始化主节点

这个操作在主节点上进行

1、初始化

kubeadm init \
--apiserver-advertise-address=192.168.110.29 \
--control-plane-endpoint=k8s-new-master \
--image-repository registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images \
--kubernetes-version v1.20.9 \
--service-cidr=10.96.0.0/16 \
--pod-network-cidr=10.244.0.0/16

注意：apiserver-advertise-address=192.168.192.8 这个后面的ip改为master的ip地址

           kubernetes-version v1.20.9，这个版本需要改成自己安装的版本

执行成功后会出现两个命令

这个直接在master节点执行就行

mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

下面这个命令是在node节点执行的，token和sha256是我自己生成的，各位看官请用自己生成的命令，此命令稍后执行

kubeadm join 192.168.110.29:6443 --token c7rkco.irtofvipbxt6wiup     --discovery-token-ca-cert-hash sha256:21ae30625bd0977c7dcfef2dd9ffc0787e479e83ead9c70ccd7ec009984b934a

2、安装网络组件

执行命令：

wget https://docs.projectcalico.org/manifests/calico.yaml

先将calico.yaml文件下载下来

这两个地方需要改下

原本是

- name: CALICO_IPV4POOL_CIDR

 value: "192.168.0.0/16"

改为

- name: CALICO_IPV4POOL_CIDR
 value: "10.244.0.0/16"

10.244.0.0/16对应的是主节点初始化的时候的 pod-network-cidr=10.244.0.0/16

然后执行命令，应用此文件。

kubectl apply -f calico.yaml

3、node加入master节点

接下来就可以在两个node节点执行join命令了，这个命令同时会让node也下载到网络组件

kubeadm join 192.168.110.29:6443 --token c7rkco.irtofvipbxt6wiup     --discovery-token-ca-cert-hash sha256:21ae30625bd0977c7dcfef2dd9ffc0787e479e83ead9c70ccd7ec009984b934a

稍等个一分钟可以在master节点执行命令 kubectl get nodes

到此时k8s集群已搭建完成！

7、dashboard可视化界面搭建

master节点执行

kubectl apply -f https://www.suibibk.com//fileupload/files/dashboard.yaml

1、下载dashboard.yaml文件

文件中增加这两个参数

修改service为NodePort类型，暴露到外部的接口为：30000

然后执行dashboard.yaml文件

kubectl apply -f dashboard.yaml

查看成功的创建信息，pod和service

kubectl get pod -n kubernetes-dashboard

kubectl get svc -n kubernetes-dashboard

2、创建kubernetes-dashboard管理员角色

创建文件dashboard-svc-account.yaml

vim dashboard-svc-account.yaml

文件内容：

apiVersion: v1
kind: ServiceAccount
metadata:
  name: dashboard-admin
  namespace: kube-system
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: dashboard-admin
subjects:
  - kind: ServiceAccount
    name: dashboard-admin
    namespace: kube-system
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io

执行文件

kubectl apply -f dashboard-svc-account.yaml

查看创建情况，执行：

kubectl -n kubernetes-dashboard get secrets

然后执行：

kubectl -n kubernetes-dashboard describe secrets admin-user-token-2f22c

可以看到结果

访问 https://192.168.110.29:30000/，192.168.110.29 是master的ip地址，30000是刚刚暴露出的端口号

将上面的token复制到页面上的位置，点击登录，效果如下

到此dashboard可视化界面安装成功！

但是这个界面全英文版，操作不如kubesphere友好

8、kubeSphere可视化界面安装

参考官方文档，在Kubernetes上安装：概述

Kubernetes 集群已配置默认 StorageClass

如果执行kubectl get sc命令效果如下

1、安装sc

先安装sc

创建sc.yaml文件，内容如下。文件中有ip地址要改

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-storage
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner
parameters:
  archiveOnDelete: "true"  ## 删除pv的时候，pv的内容是否要备份
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  labels:
    app: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/nfs-subdir-external-provisioner:v4.0.2
          # resources:
          #    limits:
          #      cpu: 10m
          #    requests:
          #      cpu: 10m
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: k8s-sigs.io/nfs-subdir-external-provisioner
            - name: NFS_SERVER
              value: 192.168.110.29 ## 指定自己nfs服务器地址
            - name: NFS_PATH  
              value: /nfs/data  ## nfs服务器共享的目录
      volumes:
        - name: nfs-client-root
          nfs:
            server: 192.168.110.29 ## 指定自己nfs服务器地址
            path: /nfs/data
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

执行命令

kubectl apply -f sc.yaml

2、安装metrics-server集群指标监控组件

创建文件metrics-server.yaml，内容如下

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    k8s-app: metrics-server
    rbac.authorization.k8s.io/aggregate-to-admin: "true"
    rbac.authorization.k8s.io/aggregate-to-edit: "true"
    rbac.authorization.k8s.io/aggregate-to-view: "true"
  name: system:aggregated-metrics-reader
rules:
- apiGroups:
  - metrics.k8s.io
  resources:
  - pods
  - nodes
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    k8s-app: metrics-server
  name: system:metrics-server
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - nodes
  - nodes/stats
  - namespaces
  - configmaps
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server-auth-reader
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: extension-apiserver-authentication-reader
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server:system:auth-delegator
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:auth-delegator
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: system:metrics-server
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:metrics-server
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
spec:
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    k8s-app: metrics-server
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
spec:
  selector:
    matchLabels:
      k8s-app: metrics-server
  strategy:
    rollingUpdate:
      maxUnavailable: 0
  template:
    metadata:
      labels:
        k8s-app: metrics-server
    spec:
      containers:
      - args:
        - --cert-dir=/tmp
        - --kubelet-insecure-tls
        - --secure-port=4443
        - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
        - --kubelet-use-node-status-port
        image: registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/metrics-server:v0.4.3
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /livez
            port: https
            scheme: HTTPS
          periodSeconds: 10
        name: metrics-server
        ports:
        - containerPort: 4443
          name: https
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /readyz
            port: https
            scheme: HTTPS
          periodSeconds: 10
        securityContext:
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          runAsUser: 1000
        volumeMounts:
        - mountPath: /tmp
          name: tmp-dir
      nodeSelector:
        kubernetes.io/os: linux
      priorityClassName: system-cluster-critical
      serviceAccountName: metrics-server
      volumes:
      - emptyDir: {}
        name: tmp-dir
---
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
  labels:
    k8s-app: metrics-server
  name: v1beta1.metrics.k8s.io
spec:
  group: metrics.k8s.io
  groupPriorityMinimum: 100
  insecureSkipTLSVerify: true
  service:
    name: metrics-server
    namespace: kube-system
  version: v1beta1
  versionPriority: 100

执行命令

kubectl apply -f metrics-server.yaml

3、安装KubeSphere

下载核心文件，分别执行两个命令，这是下载指定版本的

wget https://github.com/kubesphere/ks-installer/releases/download/v3.1.1/kubesphere-installer.yaml

wget https://github.com/kubesphere/ks-installer/releases/download/v3.1.1/cluster-configuration.yaml

如果下载最新版本那么执行：

wget https://www.suibibk.com//fileupload/files/kubesphere-installer.yaml
wget https://www.suibibk.com//fileupload/files/cluster-configuration.yaml

执行完命令后会下载两个文件kubesphere-installer.yaml，cluster-configuration.yaml

cluster-configuration.yaml文件可以设置“启用可插拔组件”。可以参考官方文档启用可插拔组件

现在不开启也行，后面可以在页面中设置

执行安装命令

kubectl apply -f kubesphere-installer.yaml

kubectl apply -f cluster-configuration.yaml

可以使用命令查看安装进度，此命令要在上面两个命令执行后一两分钟才会看到进度

kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f

如果过早执行会出现Error，不过没关系，等一会再执行一次

这是安装进度，效果如下图

安转成功会出现：

访问 http://192.168.110.29:30880，输入账号密码。

登录成功后

到此刻KubeSphere可视化界面安装成功！