1. k8s的yaml文件到底有多复杂
Kubernetes创建、更新、删除资源等操作时均可以使用json或yaml文件进行操作,更新和删除可以依赖之前的文件进行更改,但是创建具有多变形,往往编辑起来比较复杂,容器出错,而且k8s的配置项实在太多,稍微不注意就会犯错。要写好一个yaml文件,你需要了解yaml的语法,需要掌握k8s的各种配置,对于一个k8s的初学者而言,这将是一件很难的事情。
比如我们看一个同时创建一个Deployment、Service、Ingress的yaml文件内容:
[Kubernetes全栈架构师:基于世界500强的k8s实战课程](@[toc]
1. 安装说明
虽然K8s 1.20版本宣布将在1.23版本之后将不再维护dockershim,意味着K8s将不直接支持Docker,不过大家不必过于担心。一是在1.23版本之前我们仍然可以使用Docker,二是dockershim肯定会有人接盘,我们同样可以使用Docker,三是Docker制作的镜像仍然可以在其他Runtime环境中使用,所以大家不必过于恐慌。
本次安装采用的是Kubeadm安装工具,安装版本是K8s 1.20+,采用的系统为CentOS 7.9,其中Master节点3台,Node节点2台,高可用工具采用HAProxy + KeepAlived,高可用架构视频讲解点我
2. 节点规划
主机名 | IP地址 | 角色 | 配置 |
---|---|---|---|
k8s-master01 ~ 03 | 192.168.0.201 ~ 203 | Master/Worker节点 | 2C2G 40G |
k8s-node01 ~ 02 | 192.168.0.204 ~ 205 | Worker节点 | 2C2G 40G |
k8s-master-lb | 192.168.0.236 | VIP | VIP不占用机器 |
信息 | 备注 |
---|---|
系统版本 | CentOS 7.9 |
Docker版本 | 19.03.x |
K8s版本 | 1.20.x |
Pod网段 | 172.168.0.0/16 |
Service网段 | 10.96.0.0/12 |
3. 基本配置
所有节点配置hosts
[root@k8s-master01 ~]# cat /etc/hosts
192.168.0.201 k8s-master01
192.168.0.202 k8s-master02
192.168.0.203 k8s-master03
192.168.0.236 k8s-master-lb # 如果不是高可用集群,该IP为Master01的IP
192.168.0.204 k8s-node01
192.168.0.205 k8s-node02
yum源配置
curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
sed -i -e '/mirrors.cloud.aliyuncs.com/d' -e '/mirrors.aliyuncs.com/d' /etc/yum.repos.d/CentOS-Base.repo
必备工具安装
yum install wget jq psmisc vim net-tools telnet yum-utils device-mapper-persistent-data lvm2 git -y
所有节点关闭防火墙、selinux、dnsmasq、swap。服务器配置如下:
systemctl disable --now firewalld
systemctl disable --now dnsmasq
systemctl disable --now NetworkManager
setenforce 0
sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/sysconfig/selinux
sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux/config
关闭swap分区
swapoff -a && sysctl -w vm.swappiness=0
sed -ri '/^[^#]*swap/s@^@#@' /etc/fstab
安装ntpdate
rpm -ivh http://mirrors.wlnmp.com/centos/wlnmp-release-centos.noarch.rpm
yum install ntpdate -y
所有节点同步时间。时间同步配置如下:
ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
echo 'Asia/Shanghai' >/etc/timezone
ntpdate time2.aliyun.com
加入到crontab
*/5 * * * * ntpdate time2.aliyun.com
所有节点配置limit:
ulimit -SHn 65535
vim /etc/security/limits.conf
# 末尾添加如下内容
* soft nofile 655360
* hard nofile 131072
* soft nproc 655350
* hard nproc 655350
* soft memlock unlimited
* hard memlock unlimited
Master01节点免密钥登录其他节点:
ssh-keygen -t rsa
for i in k8s-master01 k8s-master02 k8s-master03 k8s-node01 k8s-node02;do ssh-copy-id -i .ssh/id_rsa.pub $i;done
下载安装所有的源码文件:
cd /root/ ; git clone https://github.com/dotbalo/k8s-ha-install.git
所有节点升级系统并重启:
yum update -y && reboot
4. 内核配置
所有节点安装ipvsadm:
yum install ipvsadm ipset sysstat conntrack libseccomp -y
所有节点配置ipvs模块
vim /etc/modules-load.d/ipvs.conf
# 加入以下内容
ip_vs
ip_vs_lc
ip_vs_wlc
ip_vs_rr
ip_vs_wrr
ip_vs_lblc
ip_vs_lblcr
ip_vs_dh
ip_vs_sh
ip_vs_fo
ip_vs_nq
ip_vs_sed
ip_vs_ftp
ip_vs_sh
nf_conntrack_ipv4
ip_tables
ip_set
xt_set
ipt_set
ipt_rpfilter
ipt_REJECT
ipip
加载内核配置
systemctl enable --now systemd-modules-load.service
开启一些k8s集群中必须的内核参数,所有节点配置k8s内核
cat <<EOF > /etc/sysctl.d/k8s.conf
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
fs.may_detach_mounts = 1
vm.overcommit_memory=1
vm.panic_on_oom=0
fs.inotify.max_user_watches=89100
fs.file-max=52706963
fs.nr_open=52706963
net.netfilter.nf_conntrack_max=2310720
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_probes = 3
net.ipv4.tcp_keepalive_intvl =15
net.ipv4.tcp_max_tw_buckets = 36000
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_max_orphans = 327680
net.ipv4.tcp_orphan_retries = 3
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.ip_conntrack_max = 65536
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_timestamps = 0
net.core.somaxconn = 16384
EOF
sysctl --system
5. 基本组件安装
所有节点安装Docker-ce 19.03
yum install docker-ce-19.03.* -y
所有节点设置开机自启动Docker
systemctl daemon-reload && systemctl enable --now docker
安装k8s组件
yum list kubeadm.x86_64 --showduplicates | sort -r
所有节点安装最新版本kubeadm
yum install kubeadm -y
默认配置的pause镜像使用gcr.io仓库,国内可能无法访问,所以这里配置Kubelet使用阿里云的pause镜像:
cat >/etc/sysconfig/kubelet<<EOF
KUBELET_EXTRA_ARGS="--pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64:3.2"
EOF
设置Kubelet开机自启动
systemctl daemon-reload
systemctl enable --now kubelet
6. 高可用组件安装
注意:如果不是高可用集群或者在云上安装,haproxy和keepalived无需安装
所有Master节点通过yum安装HAProxy和KeepAlived:
yum install keepalived haproxy -y
所有Master节点配置HAProxy(详细配置参考HAProxy文档,所有Master节点的HAProxy配置相同):
[root@k8s-master01 etc]# mkdir /etc/haproxy
[root@k8s-master01 etc]# vim /etc/haproxy/haproxy.cfg
global
maxconn 2000
ulimit-n 16384
log 127.0.0.1 local0 err
stats timeout 30s
defaults
log global
mode http
option httplog
timeout connect 5000
timeout client 50000
timeout server 50000
timeout http-request 15s
timeout http-keep-alive 15s
frontend monitor-in
bind *:33305
mode http
option httplog
monitor-uri /monitor
frontend k8s-master
bind 0.0.0.0:16443
bind 127.0.0.1:16443
mode tcp
option tcplog
tcp-request inspect-delay 5s
default_backend k8s-master
backend k8s-master
mode tcp
option tcplog
option tcp-check
balance roundrobin
default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100
server k8s-master01 192.168.0.201:6443 check
server k8s-master02 192.168.0.202:6443 check
server k8s-master03 192.168.0.203:6443 check
所有Master节点配置KeepAlived,配置不一样,注意区分
注意每个节点的IP和网卡(interface参数)
Master01节点的配置:
[root@k8s-master01 etc]# mkdir /etc/keepalived
[root@k8s-master01 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id LVS_DEVEL
script_user root
enable_script_security
}
vrrp_script chk_apiserver {
script "/etc/keepalived/check_apiserver.sh"
interval 5
weight -5
fall 2
rise 1
}
vrrp_instance VI_1 {
state MASTER
interface ens192
mcast_src_ip 192.168.0.201
virtual_router_id 51
priority 101
advert_int 2
authentication {
auth_type PASS
auth_pass K8SHA_KA_AUTH
}
virtual_ipaddress {
192.168.0.236
}
# track_script {
# chk_apiserver
# }
}
Master02节点的配置:
! Configuration File for keepalived
global_defs {
router_id LVS_DEVEL
script_user root
enable_script_security
}
vrrp_script chk_apiserver {
script "/etc/keepalived/check_apiserver.sh"
interval 5
weight -5
fall 2
rise 1
}
vrrp_instance VI_1 {
state BACKUP
interface ens192
mcast_src_ip 192.168.0.202
virtual_router_id 51
priority 100
advert_int 2
authentication {
auth_type PASS
auth_pass K8SHA_KA_AUTH
}
virtual_ipaddress {
192.168.0.236
}
# track_script {
# chk_apiserver
# }
}
Master03节点的配置:
! Configuration File for keepalived
global_defs {
router_id LVS_DEVEL
script_user root
enable_script_security
}
vrrp_script chk_apiserver {
script "/etc/keepalived/check_apiserver.sh"
interval 5
weight -5
fall 2
rise 1
}
vrrp_instance VI_1 {
state BACKUP
interface ens192
mcast_src_ip 192.168.0.203
virtual_router_id 51
priority 100
advert_int 2
authentication {
auth_type PASS
auth_pass K8SHA_KA_AUTH
}
virtual_ipaddress {
192.168.0.236
}
# track_script {
# chk_apiserver
# }
}
注意上述的健康检查是关闭的,集群建立完成后再开启:
# track_script {
# chk_apiserver
# }
配置KeepAlived健康检查文件:
[root@k8s-master01 keepalived]# cat /etc/keepalived/check_apiserver.sh
#!/bin/bash
err=0
for k in $(seq 1 3)
do
check_code=$(pgrep haproxy)
if [[ $check_code == "" ]]; then
err=$(expr $err + 1)
sleep 1
continue
else
err=0
break
fi
done
if [[ $err != "0" ]]; then
echo "systemctl stop keepalived"
/usr/bin/systemctl stop keepalived
exit 1
else
exit 0
fi
chmod +x /etc/keepalived/check_apiserver.sh
启动haproxy和keepalived
[root@k8s-master01 keepalived]# systemctl daemon-reload
[root@k8s-master01 keepalived]# systemctl enable --now haproxy
[root@k8s-master01 keepalived]# systemctl enable --now keepalived
测试VIP
[root@k8s-master01 ~]# ping 192.168.0.236 -c 4
PING 192.168.0.236 (192.168.0.236) 56(84) bytes of data.
64 bytes from 192.168.0.236: icmp_seq=1 ttl=64 time=0.464 ms
64 bytes from 192.168.0.236: icmp_seq=2 ttl=64 time=0.063 ms
64 bytes from 192.168.0.236: icmp_seq=3 ttl=64 time=0.062 ms
64 bytes from 192.168.0.236: icmp_seq=4 ttl=64 time=0.063 ms
7. 集群初始化
Master01节点创建new.yaml配置文件如下:
apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
token: 7t2weq.bjbawausm0jaxury
ttl: 24h0m0s
usages:
- signing
- authentication
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.0.201
bindPort: 6443
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: k8s-master01
taints:
- effect: NoSchedule
key: node-role.kubernetes.io/master
---
apiServer:
certSANs:
- 192.168.0.236
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: 192.168.0.236:16443
controllerManager: {}
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: 172.168.0.0/16
serviceSubnet: 10.96.0.0/12
scheduler: {}
注意:如果不是高可用集群,192.168.0.236:16443改为master01的地址,16443改为apiserver的端口,默认是6443,注意更改v1.20.0为自己服务器kubeadm的版本:kubeadm version
将new.yaml文件复制到其他master节点,之后所有Master节点提前下载镜像,可以节省初始化时间:
kubeadm config images pull --config /root/new.yaml
所有节点设置开机自启动kubelet
systemctl enable --now kubelet(如果启动失败无需管理,初始化成功以后即可启动)
Master01节点初始化,初始化以后会在/etc/kubernetes目录下生成对应的证书和配置文件,之后其他Master节点加入Master01即可:
kubeadm init --config /root/new.yaml --upload-certs
初始化成功以后,会产生Token值,用于其他节点加入时使用,因此要记录下初始化成功生成的token值(令牌值):
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
You can now join any number of the control-plane node running the following command on each as root:
kubeadm join 192.168.0.236:16443 --token 7t2weq.bjbawausm0jaxury \
--discovery-token-ca-cert-hash sha256:8c92ecb336be2b9372851a9af2c7ca1f7f60c12c68f6ffe1eb513791a1b8a908 \
--control-plane --certificate-key ac2854de93aaabdf6dc440322d4846fc230b290c818c32d6ea2e500fc930b0aa
Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.0.236:16443 --token 7t2weq.bjbawausm0jaxury \
--discovery-token-ca-cert-hash sha256:8c92ecb336be2b9372851a9af2c7ca1f7f60c12c68f6ffe1eb513791a1b8a908
Master01节点配置环境变量,用于访问Kubernetes集群:
cat <<EOF >> /root/.bashrc
export KUBECONFIG=/etc/kubernetes/admin.conf
EOF
source /root/.bashrc
查看节点状态:
[root@k8s-master01 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master01 NotReady control-plane,master 74s v1.20.0
采用初始化安装方式,所有的系统组件均以容器的方式运行并且在kube-system命名空间内,此时可以查看Pod状态:
[root@k8s-master01 ~]# kubectl get pods -n kube-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE
coredns-777d78ff6f-kstsz 0/1 Pending 0 14m <none> <none>
coredns-777d78ff6f-rlfr5 0/1 Pending 0 14m <none> <none>
etcd-k8s-master01 1/1 Running 0 14m 192.168.0.201 k8s-master01
kube-apiserver-k8s-master01 1/1 Running 0 13m 192.168.0.201 k8s-master01
kube-controller-manager-k8s-master01 1/1 Running 0 13m 192.168.0.201 k8s-master01
kube-proxy-8d4qc 1/1 Running 0 14m 192.168.0.201 k8s-master01
kube-scheduler-k8s-master01 1/1 Running 0 13m 192.168.0.201 k8s-master01
8. 高可用Master
初始化其他master加入集群
kubeadm join 192.168.0.236:16443 --token 7t2weq.bjbawausm0jaxury \
--discovery-token-ca-cert-hash sha256:8c92ecb336be2b9372851a9af2c7ca1f7f60c12c68f6ffe1eb513791a1b8a908 \
--control-plane --certificate-key ac2854de93aaabdf6dc440322d4846fc230b290c818c32d6ea2e500fc930b0aa
9. 添加Node节点
kubeadm join 192.168.0.236:16443 --token 7t2weq.bjbawausm0jaxury \
--discovery-token-ca-cert-hash sha256:8c92ecb336be2b9372851a9af2c7ca1f7f60c12c68f6ffe1eb513791a1b8a908
查看集群状态:
[root@k8s-master01]# kubectl get node
NAME STATUS ROLES AGE VERSION
k8s-master01 NotReady control-plane,master 8m53s v1.20.0
k8s-master02 NotReady control-plane,master 2m25s v1.20.0
k8s-master03 NotReady control-plane,master 31s v1.20.0
k8s-node01 NotReady <none> 32s v1.20.0
k8s-node02 NotReady <none> 88s v1.20.0
10. Calico安装
以下步骤只在master01执行
cd /root/k8s-ha-install && git checkout manual-installation-v1.20.x && cd calico/
修改calico-etcd.yaml的以下位置
sed -i 's#etcd_endpoints: "http://<ETCD_IP>:<ETCD_PORT>"#etcd_endpoints: "https://192.168.0.201:2379,https://192.168.0.202:2379,https://192.168.0.203:2379"#g' calico-etcd.yaml
ETCD_CA=`cat /etc/kubernetes/pki/etcd/ca.crt | base64 | tr -d '\n'`
ETCD_CERT=`cat /etc/kubernetes/pki/etcd/server.crt | base64 | tr -d '\n'`
ETCD_KEY=`cat /etc/kubernetes/pki/etcd/server.key | base64 | tr -d '\n'`
sed -i "s@# etcd-key: null@etcd-key: ${ETCD_KEY}@g; s@# etcd-cert: null@etcd-cert: ${ETCD_CERT}@g; s@# etcd-ca: null@etcd-ca: ${ETCD_CA}@g" calico-etcd.yaml
sed -i 's#etcd_ca: ""#etcd_ca: "/calico-secrets/etcd-ca"#g; s#etcd_cert: ""#etcd_cert: "/calico-secrets/etcd-cert"#g; s#etcd_key: "" #etcd_key: "/calico-secrets/etcd-key" #g' calico-etcd.yaml
POD_SUBNET=`cat /etc/kubernetes/manifests/kube-controller-manager.yaml | grep cluster-cidr= | awk -F= '{print $NF}'`
sed -i 's@# - name: CALICO_IPV4POOL_CIDR@- name: CALICO_IPV4POOL_CIDR@g; s@# value: "192.168.0.0/16"@ value: '"${POD_SUBNET}"'@g' calico-etcd.yaml
创建calico
kubectl apply -f calico-etcd.yaml
11. Metrics Server部署
在新版的Kubernetes中系统资源的采集均使用Metrics-server,可以通过Metrics采集节点和Pod的内存、磁盘、CPU和网络的使用率。
将Master01节点的front-proxy-ca.crt复制到所有Node节点
scp /etc/kubernetes/pki/front-proxy-ca.crt k8s-node01:/etc/kubernetes/pki/front-proxy-ca.crt
scp /etc/kubernetes/pki/front-proxy-ca.crt k8s-node(其他节点自行拷贝):/etc/kubernetes/pki/front-proxy-ca.crt
安装metrics server
cd /root/k8s-ha-install/metrics-server-0.4.x-kubeadm/
[root@k8s-master01 metrics-server-0.4.x-kubeadm]# kubectl create -f comp.yaml
serviceaccount/metrics-server created
clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created
clusterrole.rbac.authorization.k8s.io/system:metrics-server created
rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created
clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created
service/metrics-server created
deployment.apps/metrics-server created
apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created
等待kube-system命令空间下的Pod全部启动后,查看状态
[root@k8s-master01 metrics-server-0.4.x-kubeadm]# kubectl top node
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
k8s-master01 109m 2% 1296Mi 33%
k8s-master02 99m 2% 1124Mi 29%
k8s-master03 104m 2% 1082Mi 28%
k8s-node01 55m 1% 761Mi 19%
k8s-node02 53m 1% 663Mi 17%
12. Dashboard部署
cd /root/k8s-ha-install/dashboard/
[root@k8s-master01 dashboard]# kubectl create -f .
serviceaccount/admin-user created
clusterrolebinding.rbac.authorization.k8s.io/admin-user created
namespace/kubernetes-dashboard created
serviceaccount/kubernetes-dashboard created
service/kubernetes-dashboard created
secret/kubernetes-dashboard-certs created
secret/kubernetes-dashboard-csrf created
secret/kubernetes-dashboard-key-holder created
configmap/kubernetes-dashboard-settings created
role.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
deployment.apps/kubernetes-dashboard created
service/dashboard-metrics-scraper created
deployment.apps/dashboard-metrics-scraper created
在谷歌浏览器(Chrome)启动文件中加入启动参数,用于解决无法访问Dashboard的问题,参考图:
--test-type --ignore-certificate-errors
更改dashboard的svc为NodePort:
kubectl edit svc kubernetes-dashboard -n kubernetes-dashboard
将ClusterIP更改为NodePort(如果已经为NodePort忽略此步骤):
查看端口号:
根据自己的实例端口号,通过任意安装了kube-proxy的宿主机或者VIP的IP+端口即可访问到dashboard:
访问Dashboard:https://192.168.0.236:18282(请更改18282为自己的端口),选择登录方式为令牌(即token方式)
查看token值:
[root@k8s-master01 1.1.1]# kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')
Name: admin-user-token-r4vcp
Namespace: kube-system
Labels: <none>
Annotations: kubernetes.io/service-account.name: admin-user
kubernetes.io/service-account.uid: 2112796c-1c9e-11e9-91ab-000c298bf023
Type: kubernetes.io/service-account-token
Data
====
ca.crt: 1025 bytes
namespace: 11 bytes
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLXI0dmNwIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIyMTEyNzk2Yy0xYzllLTExZTktOTFhYi0wMDBjMjk4YmYwMjMiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06YWRtaW4tdXNlciJ9.bWYmwgRb-90ydQmyjkbjJjFt8CdO8u6zxVZh-19rdlL_T-n35nKyQIN7hCtNAt46u6gfJ5XXefC9HsGNBHtvo_Ve6oF7EXhU772aLAbXWkU1xOwQTQynixaypbRIas_kiO2MHHxXfeeL_yYZRrgtatsDBxcBRg-nUQv4TahzaGSyK42E_4YGpLa3X3Jc4t1z0SQXge7lrwlj8ysmqgO4ndlFjwPfvg0eoYqu9Qsc5Q7tazzFf9mVKMmcS1ppPutdyqNYWL62P1prw_wclP0TezW1CsypjWSVT4AuJU8YmH8nTNR1EXn8mJURLSjINv6YbZpnhBIPgUGk1JYVLcn47w
将token值输入到令牌后,单击登录即可访问Dashboard
K8s全栈架构师培训课程,点我了解
---
apiVersion: v1
kind: Service
metadata:
labels:
app: test-yaml
name: test-yaml
namespace: freeswitch
spec:
ports:
- name: container-1-web-1
port: 8080
protocol: TCP
targetPort: 8080
selector:
app: test-yaml
sessionAffinity: None
type: ClusterIP
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
creationTimestamp: null
name: test-yaml
spec:
rules:
- host: test.com
http:
paths:
- backend:
serviceName: test-yaml
servicePort: 8080
path: /
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: test-yaml
name: test-yaml
namespace: freeswitch
spec:
replicas: 3
selector:
matchLabels:
app: test-yaml
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
type: RollingUpdate
template:
metadata:
annotations:
info: test for yaml
labels:
app: test-yaml
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- test-yaml
topologyKey: kubernetes.io/hostname
weight: 100
containers:
- env:
- name: TZ
value: Asia/Shanghai
- name: LANG
value: C.UTF-8
image: nginx
imagePullPolicy: Always
lifecycle: {}
livenessProbe:
failureThreshold: 2
initialDelaySeconds: 30
periodSeconds: 10
successThreshold: 1
tcpSocket:
port: 8080
timeoutSeconds: 2
name: test-yaml
ports:
- containerPort: 8080
name: web
protocol: TCP
readinessProbe:
failureThreshold: 2
initialDelaySeconds: 30
periodSeconds: 10
successThreshold: 1
tcpSocket:
port: 8080
timeoutSeconds: 2
resources:
limits:
cpu: 195m
memory: 375Mi
requests:
cpu: 10m
memory: 10Mi
securityContext:
allowPrivilegeEscalation: false
capabilities: {}
privileged: false
procMount: Default
readOnlyRootFilesystem: false
runAsNonRoot: false
volumeMounts:
- mountPath: /usr/share/zoneinfo/Asia/Shanghai
name: tz-config
- mountPath: /etc/localtime
name: tz-config
- mountPath: /etc/timezone
name: timezone
dnsPolicy: ClusterFirst
hostAliases:
- hostnames:
- www.baidu.com
ip: 114.114.114.114
imagePullSecrets:
- name: myregistrykey
- name: myregistrykey2
restartPolicy: Always
securityContext: {}
volumes:
- hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
type: ""
name: tz-config
- hostPath:
path: /etc/timezone
type: ""
name: timezone
这是一个包含了Service、Ingress、Deployment比较常用并且没有用到高级功能的yaml配置,就已经有上百行,如果是在添加了一些高级配置或者是Deployment中的容器不止一个,这个yaml会更大,就会造成一种视觉上疲劳,更改起来也比较麻烦而且非常容易出错。
Kubernetes全栈架构师:基于世界500强的k8s实战课程
2. 基于图形化的方式自动生成yaml
2.1 k8s图形化管理工具Ratel安装
本次采用Ratel自动生成yaml文件,Ratel安装文档:https://github.com/dotbalo/ratel-doc/blob/master/cluster/Install.md
2.2 使用Ratel创建生成yaml文件
2.2.1 基本配置
安装完成后,可以生成、创建管理常用的k8s核心资源,比如创建一个Deployment:
点击Deployment – 创建如图所示:
之后可以填写一些基本的配置信息,比如Deployment名称、副本数、标签信息等,当然也可以点击必须/尽量部署至不同宿主机进行Pod亲和力的配置
同时也可添加一些复杂的配置,比如内核配置、容忍配置、节点亲和力快捷配置:
2.2.2 亲和力配置
基本配置编译完成以后,点击NEXT,下一个配置亲和力配置,如果上一页使用了亲和力快捷键,这边会自动生成亲和力配置,你可以再次编辑或者添加、删除:
2.2.3 存储配置
亲和力配置完成以后,可以点击NEXT进行存储配置,目前支持volume和projectedVolume配置,volume支持configMap、Secret、HostPath、PVC、NFS、Empty等常用类型的配置:
2.2.4 容器配置
接下来是容器配置,支持常用的容器配置,当然也可以添加多个容器:
稍微复杂一点的配置:
2.2.4 初始化容器配置
初始化容器和容器配置类似
2.2.5 Service和Ingress配置
创建Deployment时可以一键添加Service和Ingress,添加Service时会自动读取容器的端口配置,添加Ingress时会自动读取Service配置
2.2.6 创建资源或生成yaml文件
上述配置完成以后,可以选择创建资源或生成yaml文件,假如点击生成yaml文件,会自动生成Service、Ingress、Deployment的yaml文件,可以直接拿着使用:
生成的内容如下:
---
apiVersion: v1
kind: Service
metadata:
creationTimestamp: null
labels:
app: test-yaml
name: test-yaml
namespace: default
spec:
ports:
- name: container-1-web-1
port: 8080
protocol: TCP
targetPort: 8080
selector:
app: test-yaml
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
creationTimestamp: null
name: test-yaml
spec:
rules:
- host: test.com
http:
paths:
- backend:
serviceName: test-yaml
servicePort: 8080
path: /
status:
loadBalancer: {}
---
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: test-yaml
name: test-yaml
namespace: default
spec:
replicas: 3
selector:
matchLabels:
app: test-yaml
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
app: test-yaml
spec:
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- preference:
matchExpressions:
- key: loki
operator: In
values:
- "true"
weight: 100
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: master
operator: NotIn
values:
- "true"
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- test-yaml
topologyKey: kubernetes.io/hostname
containers:
- args:
- '*.jar --server.port=80'
command:
- java -jar
env:
- name: TZ
value: Asia/Shanghai
- name: LANG
value: C.UTF-8
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
envFrom:
- configMapRef:
name: testcm
image: nginx
imagePullPolicy: IfNotPresent
lifecycle:
postStart:
exec:
command:
- echo "start"
preStop:
exec:
command:
- sleep 30
livenessProbe:
failureThreshold: 2
initialDelaySeconds: 30
periodSeconds: 10
successThreshold: 1
tcpSocket:
port: 8080
timeoutSeconds: 2
name: test-yaml
ports:
- containerPort: 8080
name: web
protocol: TCP
readinessProbe:
failureThreshold: 2
httpGet:
httpHeaders:
- name: a
value: b
path: /
port: 8080
scheme: HTTP
initialDelaySeconds: 30
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 2
resources:
limits:
cpu: 493m
memory: 622Mi
requests:
cpu: 10m
memory: 10Mi
securityContext:
allowPrivilegeEscalation: false
capabilities: {}
privileged: false
procMount: Default
readOnlyRootFilesystem: false
runAsNonRoot: false
volumeMounts:
- mountPath: /usr/share/zoneinfo/Asia/Shanghai
name: tz-config
- mountPath: /etc/localtime
name: tz-config
- mountPath: /etc/timezone
name: timezone
- mountPath: /mnt
name: nfs-test
dnsPolicy: ClusterFirst
initContainers:
- args:
- init
command:
- echo
env:
- name: TZ
value: Asia/Shanghai
- name: LANG
value: C.UTF-8
image: nignx-init
imagePullPolicy: Always
name: init
resources:
limits:
cpu: 351m
memory: 258Mi
requests:
cpu: 10m
memory: 10Mi
securityContext:
allowPrivilegeEscalation: false
capabilities: {}
privileged: false
procMount: Default
readOnlyRootFilesystem: false
runAsNonRoot: false
volumeMounts:
- mountPath: /usr/share/zoneinfo/Asia/Shanghai
name: tz-config
- mountPath: /etc/localtime
name: tz-config
- mountPath: /etc/timezone
name: timezone
nodeSelector:
ratel: "true"
restartPolicy: Always
securityContext:
sysctls:
- name: net.core.somaxconn
value: "16384"
- name: net.ipv4.tcp_max_syn_backlog
value: "16384"
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Exists
volumes:
- name: projected-test
projected:
defaultMode: 420
sources:
- downwardAPI:
items:
- fieldRef:
fieldPath: metadata.name
path: /opt/x
- hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
type: ""
name: tz-config
- hostPath:
path: /etc/timezone
type: ""
name: timezone
- name: nfs-test
nfs:
path: /data/nfs
server: 1.1.1.1
status: {}
这个yaml比之前的稍复杂,并且添加了一些高级配置,手动编写的还是比较麻烦的,所以用Ratel自动生成还是比较方便的,并且不会出错。
3. 其他资源文件自动生成
目前支持了很多资源文件的自动生成,比如:Deployment、StatefulSet、DaemonSet、Service、Ingress、CronJob、Secret、ConfigMap、PV、PVC等,可以大大减少我们的工作量和k8s的复杂度。
如果想要系统的学习k8s,可以专注下k8s的课程:
Kubernetes全栈架构师:基于世界500强的k8s实战课程