部署要求
1、熟悉 ansible,不熟悉也没关系、按步骤操作即可
2、3台及以上机器,操作系统CentOS(7以上版本)/Ubuntu(18以上版本),配置好网络,主机名,SSH免密或SSH密码
主机名 | IP | 服务角色 |
---|---|---|
node01.example.io | 10.0.2.10 | master、node、etcd |
node02.example.io | 10.0.2.11 | node、etcd、ingress |
node03.example.io | 10.0.2.12 | node、etcd、ingress |
3、下载kubernetes和containerd二进制安装文件
k8s master 节点高可用方案
1、采用 haproxy,keepalived 实现 高可用
2、在每个kubelet 节点上运行一个 nginx的 static Pod 对 master节点进行反向代理
部署步骤
1、在ansible节点上安装ansible和git
Clone ansible脚本仓库(订阅专栏后私信,脚本问题可以一对一解答)
git clone https://gitee.com/yxydde/ansible_k8s.git
2、解压安装包 (github中有网盘链接)
sudo wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -O /usr/local/bin/cfssl
sudo wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -O /usr/local/bin/cfssljson
sudo chmod u+x /usr/local/bin/cfssl
sudo chmod u+x /usr/local/bin/cfssljson
wget https://storage.googleapis.com/kubernetes-release/release/v1.28.3/kubernetes-server-linux-amd64.tar.gz
tar -xzf kubernetes-server-linux-amd64.tar.gz
sudo mv kubernetes/server/bin/* /usr/local/bin/
wget https://github.com/etcd-io/etcd/releases/download/v3.5.9/etcd-v3.5.9-linux-amd64.tar.gz
tar -xzf etcd-v3.5.9-linux-amd64.tar.gz
sudo mv etcd-v3.5.9-linux-amd64/{etcd,etcdctl} /usr/local/bin/
sudo mkdir /opt/pkgs/
wget https://github.com/containerd/containerd/releases/download/v1.6.21/cri-containerd-cni-1.6.21-linux-amd64.tar.gz -O /opt/pkgs
# 如果是centos 7需要单独下载 Static Linking 的 runc
wget https://github.com/opencontainers/runc/releases/download/v1.1.7/runc.amd64
sudo mv runc.amd64 /usr/local/sbin/runc
sudo chmod u+x /usr/local/sbin/runc
3、编辑主机文件(配置 etcd master node 个角色的主机),规划网络
注意:k8s节点网络、service网络、Pod网络不可重叠
# cd ansible_k8s
# vi example/hosts.multi-node
[etcd]
10.0.2.10 NODE_NAME=etcd1
10.0.2.11 NODE_NAME=etcd2
10.0.2.12 NODE_NAME=etcd3
[kube_master]
10.0.2.10
[kube_node]
10.0.2.10
10.0.2.11
10.0.2.12
[ingress_node]
10.0.2.11
10.0.2.12
[kube_client]
localhost
# [optional] loadbalance for accessing k8s from outside
[ext_lb]
10.0.2.11 ROLE=MASTER PRIORITY=160
10.0.2.12 ROLE=BACKUP PRIORITY=110
[etcd:vars]
# 默认2G,建议调整到8G
ETCD_QUOTA_BACKEND_BYTES="2147483648"
[all:vars]
CONTAINERD_PKG="/opt/pkgs/cri-containerd-cni-1.6.21-linux-amd64.tar.gz"
NGINX_LOG_HOST_PATH="/var/log/nginx/"
# 配置高可用时 APISERVER 使用的负载均衡器 IP
APISERVER_LB_IP="127.0.0.1"
# apiserver HTTPS 端口
APISERVER_SECURE_PORT=6443
# apiserver 负载均衡器端口,避免和 APISERVER_SECURE_PORT 冲突
APISERVER_LB_PORT=5443
K8S_LOG_DIR="/var/log/kubernetes"
KUBELET_ROOT_DIR="/var/lib/kubelet"
CONTAINERD_ROOT_DIR="/var/lib/containerd"
# 生成CA证书文件的本地路径
PKI_GEN_DIR="{{ inventory_dir }}/pki"
YAML_GEN_DIR="{{ inventory_dir }}/yaml"
# K8S 集群名称
CLUSTER_NAME="kubernetes"
# K8S 日志级别
DEFAULT_LOG_LEVEL=1
# Service IP 网段,默认 10.96.0.0/12
SERVICE_CLUSTER_IP_RANGE="172.20.0.0/16"
# SERVICE_CLUSTER_IP_RANGE 的第1个IP
MASTER_CLUSTER_IP="172.20.0.1"
# SERVICE_CLUSTER_IP_RANGE 的第10个IP
DNS_SERVER_IP="172.20.0.10"
# Calico CIDR default 192.168.0.0/16
# Flannel CIDR default 10.244.0.0/16
CLUSTER_CIDR="10.244.0.0/16"
# PAUSE POD 镜像地址
POD_INFRA_CONTAINER_IMAGE="registry.cn-beijing.aliyuncs.com/kube-mirrors/pause:3.9"
# https://kubernetes.io/docs/reference/access-authn-authz/bootstrap-tokens/
BOOTSTRAP_TOKEN_ID="07401b"
BOOTSTRAP_TOKEN_SECRET="rany0t9iuijdk42b"
# dashboard 域名
DASHBOARD_DOMAIN="das.example.com"
ansible_ssh_user = root
ansible_ssh_pass = root
4、执行ansible脚本进行部署
# 进行系统基本设置
ansible-playbook -i example/hosts.multi-node 01.prepare.yml
# 生成相关证书文件
ansible-playbook -i example/hosts.multi-node 02.cert.yml
# 部署 etcd 集群
ansible-playbook -i example/hosts.multi-node 03.etcd.yml
# 部署containerd
ansible-playbook -i example/hosts.multi-node 04.containerd.yml
# 部署 master 节点
ansible-playbook -i example/hosts.multi-node 05.kube-master.yml
# 部署 master 节点高可用代理(可选),默认使用本地代理
ansible-playbook -i example/hosts.multi-node 06.ext-lb.yml
# 部署客户端节点
ansible-playbook -i example/hosts.multi-node 07.kube-client.yml
# 部署node节点
ansible-playbook -i example/hosts.multi-node 08.kube-node.yml
# 部署 calico 网络插件(calico/flannel二选一即可)
ansible-playbook -i example/hosts.multi-node 09.kube-calico.yml
# 部署 flannel 网络插件 (calico/flannel二选一即可)
ansible-playbook -i example/hosts.multi-node 09.kube-flannel.yml
# 部署 coredns
ansible-playbook -i example/hosts.multi-node 10.coredns.yml
# 部署 metrics-server
ansible-playbook -i example/hosts.multi-node 11.metrics-server.yml
# 部署 ingress-nginx
ansible-playbook -i example/hosts.multi-node 12.ingress-nginx.yml
# 部署 dashboard(可选)
ansible-playbook -i example/hosts.multi-node 13.dashboard.yml