kubernetes单Master集群搭建及安装UI管理界面Dashboard安装
一、环境说明
- 实验环境:
资源配置 | 说明 |
---|---|
笔记本电脑 | 内存 16G+ |
VMWare Workstation | 15+版本 |
kubernetes | 1.18.10 |
kubernetes-dashboard | 2.0.3 |
CentOS7.+ | 内核版本: 5.17.1-1.el7.elrepo.x86_64(如小于则需要进行内核升级 ) |
- 三台虚拟机内存分配如下:
节点 | 内存 | IP分配 |
---|---|---|
k8s-master | 4G (主节点,内存分配大些) | 192.168.76.140 |
k8s-node1 | 2G (Worker) | 192.168.76.141 |
k8s-node2 | 2G (Worker) | 192.168.76.142 |
如条件允许情况下,可以将所有节点内容调整高些,如4G
二、kubernetes部署方式
目前生产部署k8s集群主要有两种方式:
方式 | 说明 |
---|---|
kubeadm方式 | kubeadm是一个k8s部署工具,提供kubeadmin init和kubeadm join,用于快速部署k8s集群。kubeadm 降低部署门槛低,但是屏蔽了很多细节,遇到问题很难排查。 |
二级制包 | 从GitHub下载发行版的二进制包,手动部署每个组件,组成k8s集群。手动部署比较麻烦,但可控性高,便于学习更多原理。 |
为了快速搭建环境进行学习,我这里选择了kubeadm方式进行搭建环境。
三、kubernetes集群拓扑结构
- 集群拓扑设计图
- 集群各个节点运行的服务
注意:这个拓扑是四节点,而我们搭建的环境是三节点,比图少了一个
四、部署Kubernetes步骤
-
所有主机都需要关闭防火墙
systemctl stop firewalld systemctl disable firewalld
-
所有主机都需要关闭selinux
# 永久 sed -i 's/enforcing/disabled/' /etc/selinux/config # 重启 reboot
-
所有主机关闭swap分区
# 永久 sed -ri 's/.*swap.*/#&/' /etc/fstab # 重启 reboot
-
设置主机名
hostnamectl --static set-hostname k8s-master hostnamectl --static set-hostname k8s-node1 hostnamectl --static set-hostname k8s-node2
-
在所有主机节点上添加hosts
cat >> /etc/hosts << EOF 192.168.76.140 k8s-master 192.168.76.141 k8s-node1 192.168.76.142 k8s-node2 EOF
-
所有主机将桥接的IPv4流量传递到iptables的链
cat > /etc/sysctl.d/k8s.conf << EOF net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 vm.swappiness = 0 EOF
# 加载br_netfilter模块 modprobe br_netfilter
# 查看是否加载 lsmod | grep br_netfilter
# 生效 sysctl --system
-
所有主机时间同步
yum install ntpdate -y ntpdate time.windows.com
-
所有主机开启ipvs
yum -y install ipset ipvsadm
cat > /etc/sysconfig/modules/ipvs.modules <<EOF #!/bin/bash modprobe -- ip_vs modprobe -- ip_vs_rr modprobe -- ip_vs_wrr modprobe -- ip_vs_sh modprobe -- nf_conntrack EOF
-
授权、运行、检查是否加载
chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack
-
检查是否加载
lsmod | grep -e ipvs -e nf_conntrack
-
所有节点安装Docker/kubeadm/kubelet/kubectl
wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo
yum -y install docker-ce-18.06.3.ce-3.el7
systemctl enable docker && systemctl start docker
docker version
-
Master设置Docker镜像加速器:
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF' { "exec-opts": ["native.cgroupdriver=systemd"], "registry-mirrors": ["https://b9pmyelo.mirror.aliyuncs.com"] } EOF
sudo systemctl daemon-reload sudo systemctl restart docker
-
Matser添加阿里云的YUM软件源
cat > /etc/yum.repos.d/kubernetes.repo << EOF [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=0 repo_gpgcheck=0 gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF
-
安装kubeadm、kubelet和kubectl
yum install -y kubelet-1.18.10 kubeadm-1.18.10 kubectl-1.18.10
-
为了实现Docker使用的cgroup drvier和kubelet使用的cgroup drver一致,建议修改"/etc/sysconfig/kubelet"文件的内容(
Matser节点
):vim /etc/sysconfig/kubelet
# 修改 KUBELET_EXTRA_ARGS="--cgroup-driver=systemd"
-
设置为开机自启动即可,由于没有生成配置文件,集群初始化后自动启动:
systemctl enable kubelet
-
部署k8s的
Master
节点kubeadm init \ --apiserver-advertise-address=192.168.76.140 \ --image-repository registry.aliyuncs.com/google_containers \ --kubernetes-version v1.18.10 \ --service-cidr=10.96.0.0/12 \ --pod-network-cidr=10.244.0.0/16
执行成功后:
根据提示信息,在Master节点上使用kubectl工具:mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config
-
默认的token有效期为24小时,当过期之后,该token就不能用了,这时可以使用如下的命令创建token:
kubeadm token create --print-join-command
# 生成一个永不过期的token kubeadm token create --ttl 0
生成Token如下:20. 添加k8s的Node节点,分别在Node节点执行如下信息,其中Token是上一步的生成的值:
# 向k8s集群中添加Node节点 kubeadm join 192.168.217.100:6443 --token jffxbn.ccjd56tm99cmknk6 --discovery-token-ca-cert-hash sha256:ce2111ce594e5189255144a72268250e5eedda87470cc3a1f69f8c973927699e
-
在
Master
节点部署CNI网络插件,由于官网下载kube-flannel.yml
文件会无法访问的情况,这里新建该文件并把内容贴在如下--- apiVersion: policy/v1beta1 kind: PodSecurityPolicy metadata: name: psp.flannel.unprivileged annotations: seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default spec: privileged: false volumes: - configMap - secret - emptyDir - hostPath allowedHostPaths: - pathPrefix: "/etc/cni/net.d" - pathPrefix: "/etc/kube-flannel" - pathPrefix: "/run/flannel" readOnlyRootFilesystem: false # Users and groups runAsUser: rule: RunAsAny supplementalGroups: rule: RunAsAny fsGroup: rule: RunAsAny # Privilege Escalation allowPrivilegeEscalation: false defaultAllowPrivilegeEscalation: false # Capabilities allowedCapabilities: ['NET_ADMIN'] defaultAddCapabilities: [] requiredDropCapabilities: [] # Host namespaces hostPID: false hostIPC: false hostNetwork: true hostPorts: - min: 0 max: 65535 # SELinux seLinux: # SELinux is unused in CaaSP rule: 'RunAsAny' --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: flannel rules: - apiGroups: ['extensions'] resources: ['podsecuritypolicies'] verbs: ['use'] resourceNames: ['psp.flannel.unprivileged'] - apiGroups: - "" resources: - pods verbs: - get - apiGroups: - "" resources: - nodes verbs: - list - watch - apiGroups: - "" resources: - nodes/status verbs: - patch --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: flannel roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: flannel subjects: - kind: ServiceAccount name: flannel namespace: kube-system --- apiVersion: v1 kind: ServiceAccount metadata: name: flannel namespace: kube-system --- kind: ConfigMap apiVersion: v1 metadata: name: kube-flannel-cfg namespace: kube-system labels: tier: node app: flannel data: cni-conf.json: | { "name": "cbr0", "cniVersion": "0.3.1", "plugins": [ { "type": "flannel", "delegate": { "hairpinMode": true, "isDefaultGateway": true } }, { "type": "portmap", "capabilities": { "portMappings": true } } ] } net-conf.json: | { "Network": "10.244.0.0/16", "Backend": { "Type": "vxlan" } } --- apiVersion: apps/v1 kind: DaemonSet metadata: name: kube-flannel-ds-amd64 namespace: kube-system labels: tier: node app: flannel spec: selector: matchLabels: app: flannel template: metadata: labels: tier: node app: flannel spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: beta.kubernetes.io/os operator: In values: - linux - key: beta.kubernetes.io/arch operator: In values: - amd64 hostNetwork: true tolerations: - operator: Exists effect: NoSchedule serviceAccountName: flannel initContainers: - name: install-cni image: quay.io/coreos/flannel:v0.11.0-amd64 command: - cp args: - -f - /etc/kube-flannel/cni-conf.json - /etc/cni/net.d/10-flannel.conflist volumeMounts: - name: cni mountPath: /etc/cni/net.d - name: flannel-cfg mountPath: /etc/kube-flannel/ containers: - name: kube-flannel image: quay.io/coreos/flannel:v0.11.0-amd64 command: - /opt/bin/flanneld args: - --ip-masq - --kube-subnet-mgr resources: requests: cpu: "100m" memory: "50Mi" limits: cpu: "100m" memory: "50Mi" securityContext: privileged: false capabilities: add: ["NET_ADMIN"] env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: - name: run mountPath: /run/flannel - name: flannel-cfg mountPath: /etc/kube-flannel/ volumes: - name: run hostPath: path: /run/flannel - name: cni hostPath: path: /etc/cni/net.d - name: flannel-cfg configMap: name: kube-flannel-cfg --- apiVersion: apps/v1 kind: DaemonSet metadata: name: kube-flannel-ds-arm64 namespace: kube-system labels: tier: node app: flannel spec: selector: matchLabels: app: flannel template: metadata: labels: tier: node app: flannel spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: beta.kubernetes.io/os operator: In values: - linux - key: beta.kubernetes.io/arch operator: In values: - arm64 hostNetwork: true tolerations: - operator: Exists effect: NoSchedule serviceAccountName: flannel initContainers: - name: install-cni image: quay.io/coreos/flannel:v0.11.0-arm64 command: - cp args: - -f - /etc/kube-flannel/cni-conf.json - /etc/cni/net.d/10-flannel.conflist volumeMounts: - name: cni mountPath: /etc/cni/net.d - name: flannel-cfg mountPath: /etc/kube-flannel/ containers: - name: kube-flannel image: quay.io/coreos/flannel:v0.11.0-arm64 command: - /opt/bin/flanneld args: - --ip-masq - --kube-subnet-mgr resources: requests: cpu: "100m" memory: "50Mi" limits: cpu: "100m" memory: "50Mi" securityContext: privileged: false capabilities: add: ["NET_ADMIN"] env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: - name: run mountPath: /run/flannel - name: flannel-cfg mountPath: /etc/kube-flannel/ volumes: - name: run hostPath: path: /run/flannel - name: cni hostPath: path: /etc/cni/net.d - name: flannel-cfg configMap: name: kube-flannel-cfg --- apiVersion: apps/v1 kind: DaemonSet metadata: name: kube-flannel-ds-arm namespace: kube-system labels: tier: node app: flannel spec: selector: matchLabels: app: flannel template: metadata: labels: tier: node app: flannel spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: beta.kubernetes.io/os operator: In values: - linux - key: beta.kubernetes.io/arch operator: In values: - arm hostNetwork: true tolerations: - operator: Exists effect: NoSchedule serviceAccountName: flannel initContainers: - name: install-cni image: quay.io/coreos/flannel:v0.11.0-arm command: - cp args: - -f - /etc/kube-flannel/cni-conf.json - /etc/cni/net.d/10-flannel.conflist volumeMounts: - name: cni mountPath: /etc/cni/net.d - name: flannel-cfg mountPath: /etc/kube-flannel/ containers: - name: kube-flannel image: quay.io/coreos/flannel:v0.11.0-arm command: - /opt/bin/flanneld args: - --ip-masq - --kube-subnet-mgr resources: requests: cpu: "100m" memory: "50Mi" limits: cpu: "100m" memory: "50Mi" securityContext: privileged: false capabilities: add: ["NET_ADMIN"] env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: - name: run mountPath: /run/flannel - name: flannel-cfg mountPath: /etc/kube-flannel/ volumes: - name: run hostPath: path: /run/flannel - name: cni hostPath: path: /etc/cni/net.d - name: flannel-cfg configMap: name: kube-flannel-cfg --- apiVersion: apps/v1 kind: DaemonSet metadata: name: kube-flannel-ds-ppc64le namespace: kube-system labels: tier: node app: flannel spec: selector: matchLabels: app: flannel template: metadata: labels: tier: node app: flannel spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: beta.kubernetes.io/os operator: In values: - linux - key: beta.kubernetes.io/arch operator: In values: - ppc64le hostNetwork: true tolerations: - operator: Exists effect: NoSchedule serviceAccountName: flannel initContainers: - name: install-cni image: quay.io/coreos/flannel:v0.11.0-ppc64le command: - cp args: - -f - /etc/kube-flannel/cni-conf.json - /etc/cni/net.d/10-flannel.conflist volumeMounts: - name: cni mountPath: /etc/cni/net.d - name: flannel-cfg mountPath: /etc/kube-flannel/ containers: - name: kube-flannel image: quay.io/coreos/flannel:v0.11.0-ppc64le command: - /opt/bin/flanneld args: - --ip-masq - --kube-subnet-mgr resources: requests: cpu: "100m" memory: "50Mi" limits: cpu: "100m" memory: "50Mi" securityContext: privileged: false capabilities: add: ["NET_ADMIN"] env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: - name: run mountPath: /run/flannel - name: flannel-cfg mountPath: /etc/kube-flannel/ volumes: - name: run hostPath: path: /run/flannel - name: cni hostPath: path: /etc/cni/net.d - name: flannel-cfg configMap: name: kube-flannel-cfg --- apiVersion: apps/v1 kind: DaemonSet metadata: name: kube-flannel-ds-s390x namespace: kube-system labels: tier: node app: flannel spec: selector: matchLabels: app: flannel template: metadata: labels: tier: node app: flannel spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: beta.kubernetes.io/os operator: In values: - linux - key: beta.kubernetes.io/arch operator: In values: - s390x hostNetwork: true tolerations: - operator: Exists effect: NoSchedule serviceAccountName: flannel initContainers: - name: install-cni image: quay.io/coreos/flannel:v0.11.0-s390x command: - cp args: - -f - /etc/kube-flannel/cni-conf.json - /etc/cni/net.d/10-flannel.conflist volumeMounts: - name: cni mountPath: /etc/cni/net.d - name: flannel-cfg mountPath: /etc/kube-flannel/ containers: - name: kube-flannel image: quay.io/coreos/flannel:v0.11.0-s390x command: - /opt/bin/flanneld args: - --ip-masq - --kube-subnet-mgr resources: requests: cpu: "100m" memory: "50Mi" limits: cpu: "100m" memory: "50Mi" securityContext: privileged: false capabilities: add: ["NET_ADMIN"] env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: - name: run mountPath: /run/flannel - name: flannel-cfg mountPath: /etc/kube-flannel/ volumes: - name: run hostPath: path: /run/flannel - name: cni hostPath: path: /etc/cni/net.d - name: flannel-cfg configMap: name: kube-flannel-cfg
-
执行如下命令,使其CNI生效(
Master节点上执行
):kubectl apply -f kube-flannel.yml
执行成功如下所示:(部分截图)
查看部署CNI网络插件进度:kubectl get pods -n kube-system
执行成功如下图所示:
在Master
节点执行ifconfig
会看到如下信息:
-
在
Master
节点使用kubectl工具查看节点状态:kubectl get nodes
执行成功如下所示:
-
查看集群健康状态:
kubectl get cs
执行成功如下所示:
执行如下命令查看集群整体状态:kubectl cluster-info
执行成功如下所示:
至此集群搭建完毕
五、部署Kubernetes-dashboard步骤
-
(
Master节点上执行
)下载dashboard配置文件recommended.yaml,下载地址官网 -
修改recommended.yaml文件,修改内容如下所示:
新增type: NodePort 和 nodePort:30000,以便能实现非本机访问
:kind: Service apiVersion: v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard spec: clusterIP: 10.102.198.114 ports: - port: 443 targetPort: 8443 nodePort: 30000 selector: k8s-app: kubernetes-dashboard sessionAffinity: None # 修改 type: NodePort status: loadBalancer: {}
截图如下:
-
执行如下命令,应用并生效:
kubectl apply -f recommended.yaml
执行成功如下图所示:
使用kubectl get pods --all-namespaces
查看,成功创建;
执行如下命令查看服务运行情况:kubectl -n kubernetes-dashboard get service kubernetes-dashboard
截图所示:
-
创建认证令牌(RBAC)
- 创建一个admin-user,新增dashboard-adminuser.yaml,执行如下命令:
添加如下内容:vim dashboard-adminuser.yaml
apiVersion: v1 kind: ServiceAccount metadata: name: admin-user namespace: kubernetes-dashboard --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: admin-user roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: admin-user namespace: kubernetes-dashboard
- 使其生效,执行如下命令:
成功截图所示:kubectl apply -f dashboard-adminuser.yaml
- 创建一个admin-user,新增dashboard-adminuser.yaml,执行如下命令:
-
生成Token,执行如下命令:
kubectl -n kubernetes-dashboard describe secret $(kubectl -n kubernetes-dashboard get secret | grep admin-user | awk '{print $1}')
成功截图:
-
浏览器中输入WorkerNode节点IP地址和端口号进行访问,如
https://192.168.76.142:30000
,注意:这里一定需要使用https
协议才行
成功如下图所示:
复制之前生成的Token,然后输入,点击登录,成功如下图所示:
到此,已经全部部署完毕!!!希望对你有所帮助,谢谢点赞!!!!