主要使用ansible工具来批量管理我们需要的主机,这些主机通过接收我们编写的playbook文件来部署k8s集群,以此来实现我们的自动化部署。
ansible是新出现的自动化运维工具,基于Python开发,集合了众多运维工具(puppet、cfengine、chef、func、fabric)的优点,实现了批量系统配置、批量程序部署、批量运行命令等功能。ansible是基于模块工作的,本身没有批量部署的能力。真正具有批量部署的是ansible所运行的模块,ansible只是提供一种框架。k8s是容器集群管理系统,是一个开源的平台,可以实现容器集群的自动化部署、自动扩缩容、维护等功能。
一.准备服务器节点
ansible:192.168.184.134
master:192.168.184.102
node1: 192.168.184.103
node2: 192.168.184.104
二. 配置Ansible
1.安装ansible
配置yum源为阿里云
# cd /etc/yum.repos.d/
备份自带的yum源
# mv CentOS-Base.repo CentOS-Base.repo.backup
下载阿里云yum源
# curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
# yum clean all
# yum -y install ansible-2.9.7-1.el7.ans.noarch.rpm / yum -y install ansible
2.设置免密连接
[root@ansible ~]# ssh-keygen
[root@ansible ~]# ssh-copy-id root@192.168.184.102
[root@ansible ~]# ssh-copy-id root@192.168.184.103
[root@ansible ~]# ssh-copy-id root@192.168.184.104
2.在Ansible服务器上的/etc/hosts文件中添加k8s服务器节点信息
[root@ansible ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.184.102 master
192.168.184.103 node1
192.168.184.104 node2
[root@ansible ~]# cat /etc/redhat-release
CentOS Linux release 7.7.1908 (Core)
3.在Ansible服务器上的/etc/ansible/hosts文件中添加k8s服务器节点
[root@ansible ~]# vim /etc/ansible/hosts
[k8s-all]
192.168.184.102
192.168.184.103
192.168.184.104
[master]
192.168.184.102
[nodes]
192.168.184.103
192.168.184.104
//测试主机组长成员主机是否在线
[root@ansible ~]# ansible k8s-all -m ping
三. 修改k8s集群各节点/etc/hosts
1.创建playbook文件并执行
[root@ansible ~]# cat hosts_playbook.yml
```powershell
- hosts: nodes
remote_user: root
tasks:
- name: backup /etc/hosts
shell: mv /etc/hosts /etc/host_bak
- name: copy localhosts file to remote
copy: src=/etc/hosts dest=/etc/ owner=root group=root mode=0644
[root@ansible ~]# ansible-playbook hosts_playbook.yml
四. 安装Docker
1.创建playbook文件并执行
· 所有节点安装docker
[root@ansible ~]# cat install_docker_playbook.yml
- hosts: k8s-all
remote_user: root
vars:
docker_version: 18.09.2
tasks:
- name: install dependencies
shell: yum install -y yum-utils device-mapper-persistent-data lvm2
- name: docker-repo
shell: yum-config-manager --add-repo https://mirrors.ustc.edu.cn/docker-ce/linux/centos/docker-ce.repo
- name: install docker
yum: name=docker-ce-{{docker_version}} state=present
- name: start docker
shell: systemctl start docker && systemctl enable docker
[root@ansible ~]# vim /etc/ansible/ansible.cfg
deprecation_warnings = false ## 179默认是true,并且不生效
[root@ansible ~]# ansible-playbook install_docker_playbook.yml
五. 部署k8s master
1.开始部署之前,需要做一些初始化处理:关闭防火墙、关闭selinux、禁用swap、配置k8s阿里云yum源等,所有操作放在脚本 before.sh 中,并在2中playbook中通过script模块执行
[root@ansible ~]# cat before.sh
#!/bin/bash
#防火墙
systemctl disable firewalld
systemctl stop firewalld
setenforce 0
#禁用swap
swapoff -a
#修改内核参数
cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
#重新加载配置文件
sysctl --system
#配置阿里k8s yum源
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
#更新缓存
yum clean all -y && yum makecache -y && yum repolist -y
2.创建playbook文件,只针对master节点,安装kubectl,kubeadm,kubelet,以及flannel
yum -y install git
git clone https://gitee.com/lm_py/kube-flannel.yaml.git
[root@ansible ~]# cat deloy_master_playbook.yml
- hosts: master
remote_user: root
vars:
kube_version: 1.18.0-0
k8s_version: v1.18.0
k8s_master: 192.168.184.102
tasks:
- name: before
script: ./before.sh
- name: install kube***
yum: name={{item}} state=present
with_items:
- kubectl-{{kube_version}}
- kubeadm-{{kube_version}}
- kubelet-{{kube_version}}
- name: init k8s
shell: kubeadm init --image-repository registry.aliyuncs.com/google_containers --kubernetes-version {{k8s_version}} --apiserver-advertise-address {{k8s_master}} --pod-network-cidr=10.244.0.0/16 --token-ttl 0
- name: config kube
shell: mkdir -p $HOME/.kube && cp -i /etc/kubernetes/admin.conf $HOME/.kube/config && chown $(id -u):$(id -g) $HOME/.kube/config
- name: copy flannel yaml file
copy: src=/root/kube-flannel.yml dest=/tmp/kube-flannel.yml
- name: install flannel
shell: kubectl apply -f /tmp/kube-flannel.yml
- name: get join command
shell: kubeadm token create --print-join-command
register: join_command
- name: show join command
debug: var=join_command verbosity=0
[root@ansible ~]# vim /etc/ansible/ansible.cfg
command_warnings = False ### 187行
[root@ansible ~]# ansible-playbook deploy_master_playbook.yml
六. 部署k8s node
1.同master一样,开始部署之前,需要做一些初始化处理,所有操作放在脚本 before.sh 中,并在playbook中通过script模块执行
[root@ansible ~]# cat deloy_nodes_playbook.yml
- hosts: nodes
remote_user: root
vars:
kube_version: 1.18.0-0
tasks:
- name: before
script: ./before.sh
- name: install kube***
yum: name={{item}} state=present
with_items:
- kubeadm-{{kube_version}}
- kubelet-{{kube_version}}
- name: start kubelet
shell: systemctl enable kubelet && systemctl start kubelet
- name: join cluster
shell: kubeadm join 192.168.184.102:6443 --token hbjg13.hx2hsjibb9a02wl9 --discovery-token-ca-cert-hash sha256:ecb6cab37a34d2513700d637c27eb2868d6b35a568afd1cb331e03ab6a839e33
[root@ansible ~]# ansible-playbook deloy_nodes_playbook.yml
2.在master节点上通过kubectl get nodes看到加入到集群中的节点,并且status为Ready状态
[root@localhost ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master Ready master 21m v1.18.0
node1 Ready <none> 16m v1.18.0
node2 Ready <none> 16m v1.18.0
[root@localhost ~]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-7ff77c879f-2pcvt 1/1 Running 0 21m
coredns-7ff77c879f-lfgml 1/1 Running 0 21m
etcd-master 1/1 Running 0 21m
kube-apiserver-master 1/1 Running 0 21m
kube-controller-manager-master 1/1 Running 0 21m
kube-flannel-ds-amd64-jdr7v 1/1 Running 0 17m
kube-flannel-ds-amd64-mv79j 1/1 Running 0 18m
kube-flannel-ds-amd64-pdhfz 1/1 Running 0 17m
kube-proxy-gcm8t 1/1 Running 0 21m
kube-proxy-ngsfj 1/1 Running 0 17m
kube-proxy-rhgh2 1/1 Running 0 17m
kube-scheduler-master 1/1 Running 0 21m