使用ansible自动化部署Kubernetes

最新推荐文章于 2024-03-14 09:48:15 发布
最新推荐文章于 2024-03-14 09:48:15 发布
阅读量591 收藏
点赞数
文章标签： ansible 自动化 kubernetes
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/wang11876/article/details/132420733
版权

使用 kubeasz 部署 Kubernetes 集群

服务器列表：

IP主机名角色
192.168.100.142kube-master1,kube-master1.suosuoli.cnK8s 集群主节点 1
192.168.100.144kube-master2,kube-master2.suosuoli.cnK8s 集群主节点 2
192.168.100.146kube-master3,kube-master3.suosuoli.cnK8s 集群主节点 3
192.168.100.160node1,node1.suosuoli.cnK8s 集群工作节点 1
192.168.100.162node2,node2.suosuoli.cnK8s 集群工作节点 2
192.168.100.164node3,node3.suosuoli.cnK8s 集群工作节点 3
192.168.100.164etcd-node1,etcd-node1.suosuoli.cn集群状态存储 etcd
192.168.100.150ha1,ha1.suosuoli.cnK8s 主节点访问入口 1(高可用及负载均衡)
192.168.100.152ha2,ha2.suosuoli.cnK8s 主节点访问入口 1(高可用及负载均衡)
192.168.100.154harbor,harbor.suosuoli.cn容器镜像仓库
192.168.100.200无VIP

一. 各主机环境准备

~# apt update
~# apt install python2.7
~# ln -sv /usr/bin/python2.7 /usr/bin/python

二. 管理端安装 Ansible

2.1 安装

root@kube-master1:~# apt install ansible

2.2 Ansible 服务器基于 Key 与被控主机通讯

root@kube-master1:~# cat batch-copyid.sh
#!/bin/bash
#
# simple script to batch diliver pubkey to some hosts.
#
IP_LIST="
192.168.100.142
192.168.100.144
192.168.100.146
192.168.100.160
192.168.100.162
192.168.100.164
192.168.100.166
"

for host in ${IP_LIST}; do
    sshpass -p stevenux ssh-copy-id ${host} -o StrictHostKeyChecking=no
    if [ $? -eq 0 ]; then
        echo "copy pubkey to ${host} done."
    else
        echo "copy pubkey to ${host} failed."
    fi
done

三. 下载部署 Kubernetes 集群所需的文件

下载部署 Kubernetes 集群所需的二进制文件及 ansible playbook 和 roles，参考
集群部署前的准备

root@kube-master1:/etc/ansible# curl -C- -fLO --retry 3 https://github.com/easzlab/kubeasz/releases/download/2.2.0/easzup
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   597  100   597    0     0    447      0  0:00:01  0:00:01 --:--:--   447
100 12965  100 12965    0     0   4553      0  0:00:02  0:00:02 --:--:-- 30942
root@kube-master1:/etc/ansible# ls
ansible.cfg  easzup  hosts

root@kube-master1:/etc/ansible# chmode +x easzup

# 开始下载
root@kube-master1:/etc/ansible# ./easzup -D
[INFO] Action begin : download_all
[INFO] downloading docker binaries 19.03.5
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 60.3M  100 60.3M    0     0  1160k      0  0:00:53  0:00:53 --:--:-- 1111k
[INFO] generate docker service file
[INFO] generate docker config file
[INFO] prepare register mirror for CN
[INFO] enable and start docker
Synchronizing state of docker.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable docker
Failed to enable unit: Unit file /etc/systemd/system/docker.service is masked.
Failed to restart docker.service: Unit docker.service is masked.  # 出错，提示docker.service masked
[ERROR] Action failed : download_all   # 下载失败

# 尝试unmask，但是service文件被删除了？
root@kube-master1:/etc/ansible# systemctl unmask docker.service
Removed /etc/systemd/system/docker.service.

# 打开easzup脚本，将其生成docker.service文件的内容拷贝，自己写docker.service文件
root@kube-master1:/etc/ansible# vim easzup
...
 echo "[INFO] generate docker service file"
  cat > /etc/systemd/system/docker.service << EOF
[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.io
[Service]
Environment="PATH=/opt/kube/bin:/bin:/sbin:/usr/bin:/usr/sbin"
ExecStart=/opt/kube/bin/dockerd
ExecStartPost=/sbin/iptables -I FORWARD -s 0.0.0.0/0 -j ACCEPT
ExecReload=/bin/kill -s HUP \$MAINPID
Restart=on-failure
RestartSec=5
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
Delegate=yes
KillMode=process
[Install]
WantedBy=multi-user.target
EOF
...

# 编写docker.service启动文件
root@kube-master1:/etc/ansible# vim /etc/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.io
[Service]
Environment="PATH=/opt/kube/bin:/bin:/sbin:/usr/bin:/usr/sbin"
ExecStart=/opt/kube/bin/dockerd
ExecStartPost=/sbin/iptables -I FORWARD -s 0.0.0.0/0 -j ACCEPT
ExecReload=/bin/kill -s HUP \$MAINPID
Restart=on-failure
RestartSec=5
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
Delegate=yes
KillMode=process
[Install]
WantedBy=multi-user.target

接着下载：

root@kube-master1:/etc/ansible# ./easzup -D
[INFO] Action begin : download_all
/bin/docker
[WARN] docker is installed.
[INFO] downloading kubeasz 2.2.0
[INFO] run a temporary container
Unable to find image 'easzlab/kubeasz:2.2.0' locally
2.2.0: Pulling from easzlab/kubeasz
9123ac7c32f7: Pull complete
837e3bfc1a1b: Pull complete
Digest: sha256:a1fc4a75fde5aee811ff230e88ffa80d8bfb66e9c1abc907092abdbff073735e
Status: Downloaded newer image for easzlab/kubeasz:2.2.0
9ae404660a8e66a59ee4029d748aff96666121e847aa3667109630560e749bf8
[INFO] cp kubeasz code from the temporary container
[INFO] stop&remove temporary container
temp_easz
[INFO] downloading kubernetes v1.17.2 binaries
v1.17.2: Pulling from easzlab/kubeasz-k8s-bin
9123ac7c32f7: Already exists
fa197cdd54ac: Pull complete
Digest: sha256:d9fdc65a79a2208f48d5bf9a7e51cf4a4719c978742ef59b507bc8aaca2564f5
Status: Downloaded newer image for easzlab/kubeasz-k8s-bin:v1.17.2
docker.io/easzlab/kubeasz-k8s-bin:v1.17.2
[INFO] run a temporary container
16c0d1675f705417e662fd78871c2721061026869e5c90102c9d47c07bb63476
[INFO] cp k8s binaries
[INFO] stop&remove temporary container
temp_k8s_bin
[INFO] downloading extral binaries kubeasz-ext-bin:0.4.0
0.4.0: Pulling from easzlab/kubeasz-ext-bin
9123ac7c32f7: Already exists
96aeb45eaf70: Pull complete
Digest: sha256:cb7c51e9005a48113086002ae53b805528f4ac31e7f4c4634e22c98a8230a5bb
Status: Downloaded newer image for easzlab/kubeasz-ext-bin:0.4.0
docker.io/easzlab/kubeasz-ext-bin:0.4.0
[INFO] run a temporary container
062d302b19c21b80f02bc7d849f365c02eda002a1cd9a8e262512954f0dce019
[INFO] cp extral binaries
[INFO] stop&remove temporary container
temp_ext_bin
[INFO] downloading system packages kubeasz-sys-pkg:0.3.3
0.3.3: Pulling from easzlab/kubeasz-sys-pkg
e7c96db7181b: Pull complete
291d9a0e6c41: Pull complete
5f5b83293598: Pull complete
376121b0ab94: Pull complete
1c7cd77764e9: Pull complete
d8d58def0f00: Pull complete
Digest: sha256:342471d786ba6d9bb95c15c573fd7d24a6fd30de51049c2c0b543d09d28b5d9f
Status: Downloaded newer image for easzlab/kubeasz-sys-pkg:0.3.3
docker.io/easzlab/kubeasz-sys-pkg:0.3.3
[INFO] run a temporary container
13f40674a29b44357a551499224436eea1bae386b68138957ed93dd96bfc0f92
[INFO] cp system packages
[INFO] stop&remove temporary container
temp_sys_pkg
[INFO] downloading offline images
v3.4.4: Pulling from calico/cni
c87736221ed0: Pull complete
5c9ca5efd0e4: Pull complete
208ecfdac035: Pull complete
4112fed29204: Pull complete
Digest: sha256:bede24ded913fb9f273c8392cafc19ac37d905017e13255608133ceeabed72a1
Status: Downloaded newer image for calico/cni:v3.4.4
docker.io/calico/cni:v3.4.4
v3.4.4: Pulling from calico/kube-controllers
c87736221ed0: Already exists
e90e29149864: Pull complete
5d1329dbb1d1: Pull complete
Digest: sha256:b2370a898db0ceafaa4f0b8ddd912102632b856cc010bb350701828a8df27775
Status: Downloaded newer image for calico/kube-controllers:v3.4.4
docker.io/calico/kube-controllers:v3.4.4
v3.4.4: Pulling from calico/node
c87736221ed0: Already exists
07330e865cef: Pull complete
d4d8bb3c8ac5: Pull complete
870dc1a5d2d5: Pull complete
af40827f5487: Pull complete
76fa1069853f: Pull complete
Digest: sha256:1582527b4923ffe8297d12957670bc64bb4f324517f57e4fece3f6289d0eb6a1
Status: Downloaded newer image for calico/node:v3.4.4
docker.io/calico/node:v3.4.4
1.6.6: Pulling from coredns/coredns
c6568d217a00: Already exists
967f21e47164: Pull complete
Digest: sha256:41bee6992c2ed0f4628fcef75751048927bcd6b1cee89c79f6acb63ca5474d5a
Status: Downloaded newer image for coredns/coredns:1.6.6
docker.io/coredns/coredns:1.6.6
v2.0.0-rc3: Pulling from kubernetesui/dashboard
d8fcb18be2fe: Pull complete
Digest: sha256:c5d991d02937ac0f49cb62074ee0bd1240839e5814d6d7b51019f08bffd871a6
Status: Downloaded newer image for kubernetesui/dashboard:v2.0.0-rc3
docker.io/kubernetesui/dashboard:v2.0.0-rc3
v0.11.0-amd64: Pulling from easzlab/flannel
cd784148e348: Pull complete
04ac94e9255c: Pull complete
e10b013543eb: Pull complete
005e31e443b1: Pull complete
74f794f05817: Pull complete
Digest: sha256:bd76b84c74ad70368a2341c2402841b75950df881388e43fc2aca000c546653a
Status: Downloaded newer image for easzlab/flannel:v0.11.0-amd64
docker.io/easzlab/flannel:v0.11.0-amd64
v1.0.3: Pulling from kubernetesui/metrics-scraper
75d12d4b9104: Pull complete
fcd66fda0b81: Pull complete
53ff3f804bbd: Pull complete
Digest: sha256:40f1d5785ea66609b1454b87ee92673671a11e64ba3bf1991644b45a818082ff
Status: Downloaded newer image for kubernetesui/metrics-scraper:v1.0.3
docker.io/kubernetesui/metrics-scraper:v1.0.3
v0.3.6: Pulling from mirrorgooglecontainers/metrics-server-amd64
e8d8785a314f: Pull complete
b2f4b24bed0d: Pull complete
Digest: sha256:c9c4e95068b51d6b33a9dccc61875df07dc650abbf4ac1a19d58b4628f89288b
Status: Downloaded newer image for mirrorgooglecontainers/metrics-server-amd64:v0.3.6
docker.io/mirrorgooglecontainers/metrics-server-amd64:v0.3.6
3.1: Pulling from mirrorgooglecontainers/pause-amd64
Digest: sha256:59eec8837a4d942cc19a52b8c09ea75121acc38114a2c68b98983ce9356b8610
Status: Downloaded newer image for mirrorgooglecontainers/pause-amd64:3.1
docker.io/mirrorgooglecontainers/pause-amd64:3.1
v1.7.20: Pulling from library/traefik
42e7d26ec378: Pull complete
8a753f02eeff: Pull complete
ab927d94d255: Pull complete
Digest: sha256:5ec34caf19d114f8f0ed76f9bc3dad6ba8cf6d13a1575c4294b59b77709def39
Status: Downloaded newer image for traefik:v1.7.20
docker.io/library/traefik:v1.7.20
2.2.0: Pulling from easzlab/kubeasz
Digest: sha256:a1fc4a75fde5aee811ff230e88ffa80d8bfb66e9c1abc907092abdbff073735e
Status: Image is up to date for easzlab/kubeasz:2.2.0
docker.io/easzlab/kubeasz:2.2.0
[INFO] Action successed : download_all  # 所有镜像都下载完成

下载成功后的目录和文件：

root@kube-master1:/etc/ansible# ll
total 132
drwxrwxr-x  11 root root  4096 Feb  1 10:55 ./
drwxr-xr-x 107 root root  4096 Apr  4 22:39 ../
-rw-rw-r--   1 root root   395 Feb  1 10:35 01.prepare.yml
-rw-rw-r--   1 root root    58 Feb  1 10:35 02.etcd.yml
-rw-rw-r--   1 root root   149 Feb  1 10:35 03.containerd.yml
-rw-rw-r--   1 root root   137 Feb  1 10:35 03.docker.yml
-rw-rw-r--   1 root root   470 Feb  1 10:35 04.kube-master.yml
-rw-rw-r--   1 root root   140 Feb  1 10:35 05.kube-node.yml
-rw-rw-r--   1 root root   408 Feb  1 10:35 06.network.yml
-rw-rw-r--   1 root root    77 Feb  1 10:35 07.cluster-addon.yml
-rw-rw-r--   1 root root  3686 Feb  1 10:35 11.harbor.yml
-rw-rw-r--   1 root root   431 Feb  1 10:35 22.upgrade.yml
-rw-rw-r--   1 root root  1975 Feb  1 10:35 23.backup.yml
-rw-rw-r--   1 root root   113 Feb  1 10:35 24.restore.yml
-rw-rw-r--   1 root root  1752 Feb  1 10:35 90.setup.yml
-rw-rw-r--   1 root root  1127 Feb  1 10:35 91.start.yml
-rw-rw-r--   1 root root  1120 Feb  1 10:35 92.stop.yml
-rw-rw-r--   1 root root   337 Feb  1 10:35 99.clean.yml
-rw-rw-r--   1 root root 10283 Feb  1 10:35 ansible.cfg
drwxrwxr-x   3 root root  4096 Apr  4 22:43 bin/
drwxrwxr-x   2 root root  4096 Feb  1 10:55 dockerfiles/
drwxrwxr-x   8 root root  4096 Feb  1 10:55 docs/
drwxrwxr-x   3 root root  4096 Apr  4 22:49 down/
drwxrwxr-x   2 root root  4096 Feb  1 10:55 example/
-rw-rw-r--   1 root root   414 Feb  1 10:35 .gitignore
drwxrwxr-x  14 root root  4096 Feb  1 10:55 manifests/
drwxrwxr-x   2 root root  4096 Feb  1 10:55 pics/
-rw-rw-r--   1 root root  5607 Feb  1 10:35 README.md
drwxrwxr-x  23 root root  4096 Feb  1 10:55 roles/
drwxrwxr-x   2 root root  4096 Feb  1 10:55 tools/

root@kube-master1:/etc/ansible# ll down/
total 792416
drwxrwxr-x  3 root root      4096 Apr  4 22:49 ./
drwxrwxr-x 11 root root      4096 Feb  1 10:55 ../
-rw-------  1 root root 203646464 Apr  4 22:46 calico_v3.4.4.tar
-rw-------  1 root root  40974848 Apr  4 22:46 coredns_1.6.6.tar
-rw-------  1 root root 126891520 Apr  4 22:47 dashboard_v2.0.0-rc3.tar
-rw-r--r--  1 root root  63252595 Apr  4 22:32 docker-19.03.5.tgz
-rw-rw-r--  1 root root      1737 Feb  1 10:35 download.sh
-rw-------  1 root root  55390720 Apr  4 22:47 flannel_v0.11.0-amd64.tar
-rw-------  1 root root 152580096 Apr  4 22:49 kubeasz_2.2.0.tar
-rw-------  1 root root  40129536 Apr  4 22:48 metrics-scraper_v1.0.3.tar
-rw-------  1 root root  41199616 Apr  4 22:48 metrics-server_v0.3.6.tar
drwxr-xr-x  2 root root      4096 Oct 19 22:56 packages/
-rw-------  1 root root    754176 Apr  4 22:49 pause_3.1.tar
-rw-------  1 root root  86573568 Apr  4 22:49 traefik_v1.7.20.tar

四. 部署

4.1 编写 hosts 文件

使用/etc/ansible/example/hosts.multi-node作为参考主机清单文件:

root@kube-master1:/etc/ansible# cp example/hosts.multi-node ./hosts
root@kube-master1:/etc/ansible# cat hosts
[etcd]
192.168.100.166 NODE_NAME=etcd1

# master node(s)
[kube-master]
192.168.100.142
192.168.100.144
192.168.100.146

# work node(s)
[kube-node]
192.168.100.160
192.168.100.162
192.168.100.164

[harbor]
[ex-lb]
[chrony]
[all:vars]
CONTAINER_RUNTIME="docker"
CLUSTER_NETWORK="flannel"
PROXY_MODE="ipvs"
SERVICE_CIDR="172.20.0.0/16"
CLUSTER_CIDR="10.20.0.0/16"
NODE_PORT_RANGE="20000-40000"
CLUSTER_DNS_DOMAIN="cluster.local."
bin_dir="/usr/bin"
ca_dir="/etc/kubernetes/ssl"
base_dir="/etc/ansible"

4.2 测试所有被控机是否可达

root@kube-master1:/etc/ansible# ansible all -m ping
192.168.100.166 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
192.168.100.160 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
192.168.100.162 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
192.168.100.164 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
192.168.100.144 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
192.168.100.146 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
192.168.100.142 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

4.3 分步部署

在部署时，注意每步都验证是否安装成功。

4.3.1 部署前准备

roles/deploy:

roles/deploy/
├── defaults
│   └── main.yml		# 配置文件：证书有效期，kubeconfig 相关配置
├── files
│   └── read-group-rbac.yaml	# 只读用户的 rbac 权限配置
├── tasks
│   └── main.yml		# 主任务脚本
└── templates
    ├── admin-csr.json.j2	# kubectl客户端使用的admin证书请求模板
    ├── ca-config.json.j2	# ca 配置文件模板
    ├── ca-csr.json.j2		# ca 证书签名请求模板
    ├── kube-proxy-csr.json.j2  # kube-proxy使用的证书请求模板
    └── read-csr.json.j2        # kubectl客户端使用的只读证书请求模板

root@kube-master1:/etc/ansible# ansible-playbook 01.prepare.yml
...

4.3.2 部署 etcd 集群

roles/etcd/:

root@kube-master1:/etc/ansible# tree roles/etcd/
roles/etcd/
├── clean-etcd.yml
├── defaults
│   └── main.yml
├── tasks
│   └── main.yml
└── templates
    ├── etcd-csr.json.j2   # 可修改证书相应信息后部署
    └── etcd.service.j2

root@kube-master1:/etc/ansible# ansible-playbook 02.etcd.yml
...

在任意 etcd 节点验证 etcd 运行信息：

root@etcd-node1:~# systemctl status etcd
root@etcd-node1:~# journalctl -u etcd

# 或者使用下面的命令查看节点健康状态
root@etcd-node1:~# apt install etcd-client
root@etcd-node1:~# for ip in ${NODE_IPS}; do ETCDCTL_API=3 etcdctl --endpoints=https://${ip}:2379 --cacert=/etc/kubernetes/ssl/ca.pem --cert=/etc/etcd/ssl/etcd.pem --key=/etc/etcd/ssl/etcd-key.pem   endpoint health; done
https://192.168.100.166:2379 is healthy: successfully committed proposal: took = 1.032861ms

4.3.3 部署 Docker

roles/docker/:

root@kube-master1:/etc/ansible# tree roles/docker/
roles/docker/
├── defaults
│   └── main.yml		# 变量配置文件
├── files
│   ├── docker			# bash 自动补全
│   └── docker-tag		# 查询镜像tag的小工具
├── tasks
│   └── main.yml		# 主执行文件
└── templates
    ├── daemon.json.j2		# docker daemon 配置文件
    └── docker.service.j2	# service 服务模板

root@kube-master1:/etc/ansible# ansible-playbook 03.docker.yml
...

部署成功后验证信息：

~# systemctl status docker 	# 服务状态
~# journalctl -u docker 		# 运行日志
~# docker version
~# docker info

4.3.4 部署 Master 节点

部署 master 节点主要包含三个组件 apiserver scheduler controller-manager，其中：

  • apiserver 提供集群管理的 REST API 接口，包括认证授权、数据校验以及集群状态变更等
    • 只有 API Server 才直接操作 etcd
    • 其他模块通过 API Server 查询或修改数据
    • 提供其他模块之间的数据交互和通信的枢纽
  • scheduler 负责分配调度 Pod 到集群内的 node 节点
    • 监听 kube-apiserver，查询还未分配 Node 的 Pod
    • 根据调度策略为这些 Pod 分配节点
  • controller-manager 由一系列的控制器组成，它通过 apiserver 监控整个集群的状态，
    并确保集群处于预期的工作状态

roles/kube-master/:

root@kube-master1:/etc/ansible# tree roles/kube-master/
roles/kube-master/
├── defaults
│   └── main.yml
├── tasks
│   └── main.yml
└── templates
    ├── aggregator-proxy-csr.json.j2
    ├── basic-auth.csv.j2
    ├── basic-auth-rbac.yaml.j2
    ├── kube-apiserver.service.j2
    ├── kube-controller-manager.service.j2
    ├── kubernetes-csr.json.j2
    └── kube-scheduler.service.j2

部署：

root@kube-master1:/etc/ansible# ansible-playbook 04.kube-master.yml
...

部署完成后查看 master 节点信息：

root@kube-master1:/etc/ansible# kubectl get nodes -o wide
NAME              STATUS                     ROLES    AGE     VERSION   INTERNAL-IP       EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
192.168.100.142   Ready,SchedulingDisabled   master   4m17s   v1.17.2   192.168.100.142   <none>        Ubuntu 18.04.3 LTS   4.15.0-55-generic   docker://19.3.5
192.168.100.144   Ready,SchedulingDisabled   master   4m18s   v1.17.2   192.168.100.144   <none>        Ubuntu 18.04.3 LTS   4.15.0-55-generic   docker://19.3.5
192.168.100.146   Ready,SchedulingDisabled   master   4m17s   v1.17.2   192.168.100.146   <none>        Ubuntu 18.04.3 LTS   4.15.0-55-generic   docker://19.3.5

4.3.5 部署 node 节点

roles/kube-node/:

root@kube-master1:/etc/ansible# tree roles/kube-node/
roles/kube-node/
├── defaults
│   └── main.yml
├── tasks
│   ├── create-kubelet-kubeconfig.yml
│   ├── main.yml
│   ├── node_lb.yml
│   └── offline.yml
└── templates
    ├── cni-default.conf.j2
    ├── haproxy.cfg.j2
    ├── haproxy.service.j2
    ├── kubelet-config.yaml.j2
    ├── kubelet-csr.json.j2
    ├── kubelet.service.j2
    └── kube-proxy.service.j2

部署：

root@kube-master1:/etc/ansible# ansible-playbook 05.kube-node.yml
...

查看全部节点信息：

root@kube-master1:/etc/ansible# kubectl get nodes -o wide
NAME              STATUS                     ROLES    AGE     VERSION   INTERNAL-IP       EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
192.168.100.142   Ready,SchedulingDisabled   master   6m32s   v1.17.2   192.168.100.142   <none>        Ubuntu 18.04.3 LTS   4.15.0-55-generic   docker://19.3.5
192.168.100.144   Ready,SchedulingDisabled   master   6m33s   v1.17.2   192.168.100.144   <none>        Ubuntu 18.04.3 LTS   4.15.0-55-generic   docker://19.3.5
192.168.100.146   Ready,SchedulingDisabled   master   6m32s   v1.17.2   192.168.100.146   <none>        Ubuntu 18.04.3 LTS   4.15.0-55-generic   docker://19.3.5
192.168.100.160   Ready                      node     31s     v1.17.2   192.168.100.160   <none>        Ubuntu 18.04.3 LTS   4.15.0-55-generic   docker://19.3.5
192.168.100.162   Ready                      node     31s     v1.17.2   192.168.100.162   <none>        Ubuntu 18.04.3 LTS   4.15.0-55-generic   docker://19.3.5
192.168.100.164   Ready                      node     31s     v1.17.2   192.168.100.164   <none>        Ubuntu 18.04.3 LTS   4.15.0-55-generic   docker://19.3.5

4.3.6 部署网络 flannel

roles/flannel/:

root@kube-master1:/etc/ansible# tree roles/flannel/
roles/flannel/
├── defaults
│   └── main.yml
├── tasks
│   └── main.yml
└── templates
    └── kube-flannel.yaml.j2

# 确保变量CLUSTER_NETWORK值为flannel
root@kube-master1:/etc/ansible# grep "CLUSTER_NETWORK" hosts
CLUSTER_NETWORK="flannel"

部署：

root@kube-master1:/etc/ansible# ansible-playbook 06.network.yml
...

部署完成后查看 flannel 是否都运行：

root@kube-master1:/etc/ansible# kubectl get pods -A -o wide
NAMESPACE     NAME                          READY   STATUS    RESTARTS   AGE   IP                NODE              NOMINATED NODE   READINESS GATES
kube-system   kube-flannel-ds-amd64-8k8jz   1/1     Running   0          29s   192.168.100.160   192.168.100.160   <none>           <none>
kube-system   kube-flannel-ds-amd64-f76vp   1/1     Running   0          29s   192.168.100.164   192.168.100.164   <none>           <none>
kube-system   kube-flannel-ds-amd64-hpgn8   1/1     Running   0          29s   192.168.100.142   192.168.100.142   <none>           <none>
kube-system   kube-flannel-ds-amd64-j2k8x   1/1     Running   0          29s   192.168.100.144   192.168.100.144   <none>           <none>
kube-system   kube-flannel-ds-amd64-spqg8   1/1     Running   0          29s   192.168.100.162   192.168.100.162   <none>           <none>
kube-system   kube-flannel-ds-amd64-strfx   1/1     Running   0          29s   192.168.100.146   192.168.100.146   <none>           <none>

4.3.7 安装集群插件

集群插件安装角色定义了: cube-dns、metric server、dashboard 及 ingress 的安装。

roles/cluster-addon/:

root@kube-master1:/etc/ansible# tree roles/cluster-addon/
roles/cluster-addon/
├── defaults
│   └── main.yml
├── tasks
│   ├── ingress.yml
│   └── main.yml
└── templates
    ├── coredns.yaml.j2
    ├── kubedns.yaml.j2
    └── metallb
        ├── bgp.yaml.j2
        ├── layer2.yaml.j2
        └── metallb.yaml.j2

部署：

root@kube-master1:/etc/ansible# ansible-playbook 07.cluster-addon.yml
...

部署完成后查看相关的 service：

root@kube-master1:/etc/ansible# kubectl get svc -A
NAMESPACE     NAME                        TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                       AGE
default       kubernetes                  ClusterIP   172.20.0.1      <none>        443/TCP                       159m
kube-system   dashboard-metrics-scraper   ClusterIP   172.20.36.20    <none>        8000/TCP                      108m
kube-system   kube-dns                    ClusterIP   172.20.0.2      <none>        53/UDP,53/TCP,9153/TCP        108m
kube-system   kubernetes-dashboard        NodePort    172.20.230.96   <none>        443:35964/TCP                 108m
kube-system   metrics-server              ClusterIP   172.20.146.16   <none>        443/TCP                       108m
kube-system   traefik-ingress-service     NodePort    172.20.69.230   <none>        80:23456/TCP,8080:23823/TCP   108m

4.4 创建 Pod 资源测试网络

测试 Pod 资源配置清单：

root@kube-master1:/etc/ansible# cat /opt/k8s-data/pod-ex.yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod
  labels:
    app: nginx
spec:
  containers:
  - name: nginx-pod
    image: nginx:1.16.1

创建该 Pod 资源：

root@kube-master1:/etc/ansible# kubectl apply -f /opt/k8s-data/pod-ex.yaml
pod/nginx-pod created
root@kube-master1:/etc/ansible# kubectl get -f /opt/k8s-data/pod-ex.yaml -w
NAME        READY   STATUS    RESTARTS   AGE
nginx-pod   1/1     Running   0          8s

验证网络连通性：

root@kube-master1:/etc/ansible# kubectl exec -it nginx-pod bash
root@nginx-pod:/# apt update
...
root@nginx-pod:/# apt install iputils-ping
...

root@nginx-pod:/# ping 172.20.0.1
PING 172.20.0.1 (172.20.0.1) 56(84) bytes of data.
64 bytes from 172.20.0.1: icmp_seq=1 ttl=64 time=0.041 ms
64 bytes from 172.20.0.1: icmp_seq=2 ttl=64 time=0.044 ms
^C
--- 172.20.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 21ms
rtt min/avg/max/mdev = 0.041/0.042/0.044/0.006 ms

root@nginx-pod:/# ping 192.168.100.142
PING 192.168.100.142 (192.168.100.142) 56(84) bytes of data.
64 bytes from 192.168.100.142: icmp_seq=1 ttl=63 time=0.296 ms
64 bytes from 192.168.100.142: icmp_seq=2 ttl=63 time=0.543 ms
^C
--- 192.168.100.142 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 8ms
rtt min/avg/max/mdev = 0.296/0.419/0.543/0.125 ms

root@nginx-pod:/# ping www.baidu.com
PING www.a.shifen.com (104.193.88.123) 56(84) bytes of data.
64 bytes from 104.193.88.123: icmp_seq=4 ttl=127 time=211 ms
64 bytes from 104.193.88.123: icmp_seq=5 ttl=127 time=199 ms
^C
--- www.a.shifen.com ping statistics ---
6 packets transmitted, 2 received, 66.6667% packet loss, time 301ms
rtt min/avg/max/mdev = 199.135/205.145/211.156/6.027 ms

root@nginx-pod:/# traceroute www.baidu.com
traceroute to www.baidu.com (104.193.88.123), 30 hops max, 60 byte packets
 1  10.20.4.1 (10.20.4.1)  0.036 ms  0.008 ms  0.007 ms
 2  192.168.100.2 (192.168.100.2)  0.083 ms  0.163 ms  0.191 ms  # 虚拟机的网关
 3  * * *
 4  * * *
 5  * * *
 6  *^C

4.5 测试 Dashboard

拿到登录 token：

root@kube-master1:/etc/ansible# kubectl get secret -A | grep dashboard-token
kube-system       kubernetes-dashboard-token-qcb5j                 kubernetes.io/service-account-token   3      105m
root@kube-master1:/etc/ansible# kubectl describe secret kubernetes-dashboard-token-qcb5j -n kube-system
Name:         kubernetes-dashboard-token-qcb5j
Namespace:    kube-system
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: kubernetes-dashboard
              kubernetes.io/service-account.uid: 00ba4b90-b914-4e5f-87dd-a808542acc83

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1346 bytes
namespace:  11 bytes
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IkRWWU45SmJEWTJZRmFZWHlEdFhhOGhKRzRWYkFEZ2M3LTU1TmZHQVlSRW8ifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC10b2tlbi1xY2I1aiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjAwYmE0YjkwLWI5MTQtNGU1Zi04N2RkLWE4MDg1NDJhY2M4MyIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTprdWJlcm5ldGVzLWRhc2hib2FyZCJ9.AR41PMi6jk_tzPX0Z88hdI6eEvldqa0wOnzJFciMa-hwnG3214dyROGwXoKtLvmshk8hjE2W11W9VBPQihF2kWqAalSQLF4WsX5LTL0X_FPO6EbYmPxUysFU9nWWRiw1UZD9WkBPl_JfLoz5LUVvrJm2fJJ80zIcrVEQ75c_Zxsgfl1k8BDMCFqcNDUiMCiJCuu-02aqdvMvXf7EYciOzqp36_wZTi9RNLEW7fZrnRu6LCIfn6XFn43nLnJuCupUZhqcSMVS0t8Qj5AY-XJRpyDamSseaF4gRwm7CixUq03Fdk8_5Esz16w2VNlZ0GINyGJTYmg8WOtQGMX9eoWHaA
墨烦信息
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 0
    评论
Kubernetes(8)-使用ansible自动化部署Kubernetes
Blog of Stevenux
04-07 1233
使用 kubeasz 部署 Kubernetes 集群，kubeasz 项目地址 服务器列表： IP 主机名 角色 192.168.100.142 kube-master1,kube-master1.suosuoli.cn K8s 集群主节点 1 192.168.100.144 kube-master2,kube-master2.suosuoli.cn K8s 集群主节点 2 ...
docker.service启动失败：Unit not found
weixin_33937499的博客
04-20 3355
因为最近一直在折腾Kubernetes集群版本升级、Docker版本升级，所以不停的把测试环境安装、还原、升级、降级，简直乱的不行。终于，在测试Docker版本升级后，启动Docker时，遇到了docker.service: Unit not found。问题虽然不大，但是却折磨了我几个小时，所以在此mark一下。 操作系统：Red Hat Enterprise Linux 7 原因1：docke...
tomcat常见错误及解决方案
weixin_34235105的博客
11-29 1016
我们习惯的web容器都是tomcat,但是tomcat的内核很丰富，如果我们要完全理解他，需要很长的时间和有较多的编程经验。 不管是初学者还是有经验的web程序员，使用Tomcat容器的时候会发生包括404，运行启动错误，以及乱码等等的问题，现在我将开发过程中遇到的常见错误和简单解决方法贴出来供大家看看，里面有部分是同事总结，我们共同完成的这篇文章。。。一，to...
Kubernetes 二进制部署 《easzlab / kubeasz项目部署》- 06-安装网络组件
u010230019的博客
03-07 1048
把本机网卡的所有IP列出来，然后选择第一个IP（docker0与lo网卡的IP会被忽略掉），第一个IP会因为操作系统的不同而不同，比如在centos操作系统中，假设网卡ens33有一个IP 192.168.92.101，网卡ens37有两个IP 192.168.90.101(primray)和192.168.90.10(second)，那么找到的IP就是192.168.90.101（好像是按每个网卡的primary IP升序排列）如果一个网卡有多个IP，都可以到达x.x.x.x，则选择哪一个（待验证）。
docker报Unable to find image ‘image-name:tag‘ locallydocker: Error response from daemon: pull acces
liuqingup的专栏
07-13 1万+
在使用Docker时，遇到拉取镜像失败的问题。其中一个错误信息是"Unable to find image ‘image-name:tag’ locally docker: Error response from daemon: pull access denied for image-name, repository does not exist or may require ‘docker login’: denied: requested access to the resource is denie
Ansible自动化部署K8S集群（包含Ansible，Playbook的使用）
02-17
Ansible自动化部署Kubernetes，超级简单，几分钟就可以部署完成，有部署过程及源代码，Ansible的简单使用以及playbook的使用例子，有完整代码
使用 ansible 部署 kubernetes 二进制文件
03-04
在 IT 运维领域，自动化部署工具的使用已经成为提高效率、减少错误的关键。Ansible 是一个流行的开源自动化工具，常用于配置管理、应用部署、任务执行等场景。本篇文章将详细探讨如何使用 Ansible 部署 Kubernetes ...
Ansible-ansible-kubernetes-module.zip
09-18
Ansible-ansible-kubernetes-module.zip,用于设置kubernetes对象的模块sible kubernetes模块,ansible是一个简单而强大的自动化引擎。它用于帮助配置管理、应用程序部署和任务自动化。
kubeadm-ansible：通过Ansible使用kubeadm构建Kubernetes集群
01-31
"kubeadm-ansible：通过Ansible使用kubeadm构建Kubernetes集群"是一个旨在利用Ansible自动化工具来简化Kubernetes集群搭建过程的项目。在这个项目中，我们将深入探讨Ansible与kubeadm的结合使用，以及它们如何帮助...
ansible-role-kubernetes：Ansible角色-Kubernetes
01-31
总结，ansible-role-kubernetes为Kubernetes的自动化部署提供了强大的支持，通过Ansible的角色机制，我们可以更加高效地管理和维护Kubernetes集群。无论是小型开发环境还是大规模生产环境，都能从中受益，实现更加...
kubernetes-ansible：使用Ansible将Kubeadm步骤转换为二进制安装步骤
02-06
Kubernetes-Ansible 该存储库正在使用Ansible部署二进制高可用性Kubernetes集群。 角色步骤使用官方部署工具Kubeadm。 高可用性 在127.0.0.1:8443上运行envoy负载平衡器的每个节点都代理kube-apiserver服务。 upstream +-------------------+ +-------------------------- > | kube-master1:6443 |
ansible-playbook一体化部署k8s前置准备工作(升级内核+时间同步+hostname/hosts配置+关闭防火墙|selinux+关闭swap+安装docker)
最新发布
Henry的博客
03-14 578
在安装k8s前需要一系列的前置准备工作，并且集群化所有节点服务器都需要做同样的操作，工作麻烦，使用ansible-playbook可以流程化这些前置工作。
Ubuntu 20.04 - 解决 Failed to enable unit: Unit file sshd.service does not exist.
热门推荐
qq_29761395的博客
02-19 2万+
文章目录起因解决方法参考 起因 事情是这样的，原本 SSH 服务是可用的： $ sudo systemctl status sshd [sudo] password for mk: ● ssh.service - OpenBSD Secure Shell server Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled) Active: active (running) sin
Ansible自动化部署高可用K8S集群
qq_42409495的博客
12-08 2388
1.1 Ansible介绍 Ansible 是一种IT自动化工具。它可以配置系统，部署软件以及协调更高级的IT任务，例如持续部署，滚动更新。Ansible 适用于管理企业IT基础设施，从具有少数主机的小规模到数千个实例的企业环境。Ansible 也是一种简单的自动化语言，可以完美地描述IT应用程序基础结构。 具备以下三个特点： 简单：减少学习成本 强大：协调应用程序生命周期 无代理：可预测，可靠和安全 使用文档：https://releases.ansible.com/ansible/ 安装 A
ansible自动化部署k8s集群
weixin_50801368的博客
06-19 696
| 192.168.1.151 | master | | 192.168.1.152 | node01| | 192.168.1.152 | node03 |
【经验分享】【Github】Error: Action failed with “The process ‘/usr/bin/git‘ failed with exit code 128“
以放的博客
02-12 998
首先打开Actions报错的仓库，找到仓库下的settings->Actions->General。返回Actions，重新部署即可成功。，勾选下面的选项即可。
Mac安装Docker和K8S
幸运小侯子
10-29 1808
Docker安装 1.brew安装docker ruishens-MacBook-Pro:docker ruishen$ brew install --cask --appdir=/Applications docker ==> Downloading https://desktop.docker.com/mac/main/amd64/69879/Docker.dmg ###########################################################...
ansible一键部署k8s（自动化）
weixin_48753359的博客
11-23 7783
ANSIBLE（自动化）一键部署k8s 主要使用ansible工具来批量管理我们需要的主机，这些主机通过接收我们编写的playbook文件来部署k8s集群，以此来实现我们的自动化部署。 ansible是新出现的自动化运维工具，基于Python开发，集合了众多运维工具（puppet、cfengine、chef、func、fabric）的优点，实现了批量系统配置、批量程序部署、批量运行命令等功能。ansible是基于模块工作的，本身没有批量部署的能力。真正具有批量部署的是ansible所运行的模块，ansib
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

墨烦信息

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值