使用rke部署集群时,虽然已经很方便了,但是如果节点个数较多,每个节点要手动配置,防火墙,selinux, iptables等等。还是挺麻烦的, 这些配置每个节点都一样,完全可以通过脚本批量配置。
过程参考RKE部署K8S
脚本主要分为四个部分:
- 批量配置服务器环境
- 配置免密登录
- 自动生成cluster.yml文件
- 下载安装RKE,部署集群。
主要使用的工具就是expecet, 关于expcet的使用参考expect用法
在配置固定时, 我们只需要了解,节点的ip, 账号及密码。 先将这些信息写入ip.txt
[root@VM-16-15-centos ~]# cat ip.txt
10.206.16.8 root 123456
10.206.16.4 root 123456
10.206.16.7 root 123456
10.206.16.9 root 123456
10.206.16.2 root 123456
脚本如下:
#!/bin/bash
help ()
{
echo ' ================================================================ '
echo ' 脚本没有参数,请在ip.txt中写入ip地址,用户及其密码'
echo ' 如果rke或kubectl下载失败请手动下载或者翻墙或者国外节点'
echo ' ================================================================'
}
case "$1" in
-h|--help) help; exit;;
esac
rke_user="rke"
rpm -qa | grep "expect"
if [ $? -eq 0 ]
then
echo "expect installed"
else
yum install -y expect
fi
cat ip.txt | while read line #使用while命令循环登录主机进行配置
do
ip=`echo ${line}|awk '{print $1}'` #ip地址
user=`echo ${line}|awk '{print $2}'`
passwd=`echo ${line}|awk '{print $3}'`
set timeout 30
expect << EOF
spawn ssh $user@$ip
expect {
"yes/no" { send "yes\r";exp_continue }
"password" { send "$passwd\r";exp_continue }
"already installed" { send "\r"}
}
# expect "yes/no"; send "yes\r"
# expect "password"; send "$passwd\r"
# 基本配置
expect "*root" ; send "hostnamectl set-hostname node${ip: 6}\r"
# 配置hostname
expect "*root" ; send "systemctl stop firewalld\r"
# 关闭防火墙
expect "*root" ; send "systemctl disable firewalld\r"
# 关闭防火墙自启动
expect "*root" ; send "echo \`date +\"%Y-%m-%d %H:%M:%S\"\` firewalld diabled > /root/.k8s.log\r"
# 日志
expect "*root" ; send "swapoff -a\r"
# 关闭swap
expect "*root" ; send "sed -ri 's/.*swap.*/#&/' /etc/fstab\r"
# 永久关闭swap
expect "*root" ; send "echo \`date +\"%Y-%m-%d %H:%M:%S\"\` swap diabled >> /root/.k8s.log\r"
# 日志
expect "*root" ; send "setenforce 0\r"
# 关闭selinux
expect "*root" ; send "sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config\r"
# 永久关闭selinux
# expect "*root" ; send "yum install ntpdate -y\r " # 安装ntpdate
expect "*root" ; send "ntpdate time.windows.com\r"
# 时间同步
expect "*root" ; send "cat >> /etc/sysctl.conf<<EOF
net.ipv4.ip_forward=1
net.bridge.bridge-nf-call-iptables=1
net.ipv4.neigh.default.gc_thresh1=4096
net.ipv4.neigh.default.gc_thresh2=6144
net.ipv4.neigh.default.gc_thresh3=8192
EOF\r"
# kernel性能调优
expect "*root" ; send "cp /etc/sysctl.conf /etc/sysctl.conf_copy\r"
expect "*root" ; send "sort -n /etc/sysctl.conf_copy | uniq >/etc/sysctl.conf\r"
expect "*root" ; send "rm -f /etc/sysctl.conf_copy\r"
expect "*root" ; send "sysctl -p\r"
# 加载配置文件
expect "*root" ; send "yum -y install lrzsz\r"
# expect "*root" ; send "yum -y install lrzsz vim gcc glibc openssl openssl-devel net-tools wget curl\r"
# 安装基础软件包
expect "*root" ; send "echo \"test soft\" >> test.txt\r"
# 安装docker
expect "*root" ; send "install -y yum-utils device-mapper-persistent-data lvm2\r"
expect "*root" ; send "yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo\r"
expect "*root" ; send "yum -y install docker-ce-18.09.9-3.el7 docker-ce-cli-18.09.9-3.el7\r"
expect "*root" ; send "systemctl start docker\r"
expect "*root" ; send "cat > /etc/docker/daemon.json <<EOF
{
\"oom-score-adjust\": -1000,
\"registry-mirrors\": \[\"https://7bezldxe.mirror.aliyuncs.com/\",\"https://kw88y6eh.mirror.aliyuncs.com\"\],
\"storage-driver\": \"overlay2\",
\"storage-opts\": \[\"overlay2.override_kernel_check=true\"\]
}
EOF\r"
expect "*root" ; send "echo \`date +\"%Y-%m-%d %H:%M:%S\"\` docker installed >> /root/.k8s.log\r"
# 日志
# 创建rke用户
expect "*root" ; send "useradd $rke_user -G docker\r"
expect "*root" ; send "echo \"123456\" | passwd --stdin $rke_user\r"
expect "*root" ; send "echo \`date +\"%Y-%m-%d %H:%M:%S\"\` user $rke_user created >> /root/.k8s.log\r"
expect "*root" ; send "exit\r"
expect eof
EOF
done
# 判断id_rsa密钥文件是否存在, 没有则创建
rke_user="rke"
if [ ! -f /home/$rke_user/.ssh/id_rsa ];then
runuser -l $rke_user -c "ssh-keygen -t rsa -P \"\" -f /home/$rke_user/.ssh/id_rsa"
else
echo "id_rsa has created ..."
fi
#密钥分发到各个节点
while read line
do
ip=`echo $line | cut -d " " -f 1`
expect <<EOF
set timeout 30
spawn su rke
expect "rke"; send "ssh-copy-id -i /home/$rke_user/.ssh/id_rsa.pub $rke_user@$ip\r"
expect {
"yes/no" { send "yes\r";exp_continue }
"password" { send "123456\r";exp_continue }
"WARNING" {send "exit\r"}
}
EOF
done < ip.txt
# 创建cluster.yml
path_cluster_yml="/home/$rke_user/cluster.yml"
if [ ! -e $path_cluster_yml ]
then
cat > $path_cluster_yml << EOF
nodes:
EOF
while read line
do
ip=`echo $line | cut -d " " -f 1`
cat >> $path_cluster_yml << EOF
- address: $ip
internal_address: $ip
user: $rke_user
role: [controlplane,worker,etcd]
EOF
done < ip.txt
cat >> $path_cluster_yml << EOF
services:
etcd:
snapshot: true
creation: 6h
retention: 24h
EOF
fi
# 安装rke
if [ -e /usr/bin/rke ]
then
echo "rke already installed"
else
wget https://github.com/rancher/rke/releases/download/v1.3.0/rke_linux-amd64
chmod +x rke_linux-amd64
mv rke_linux-amd64 /usr/bin/rke
fi
# 安装kubectl
if [ -e /usr/local/bin/kubectl ]
then
echo "kubectl already installed"
else
wget http://rancher-mirror.cnrancher.com/kubectl/v1.21.4/linux-amd64-v1.21.4-kubectl
mv linux-amd64-v1.21.4-kubectl /usr/local/bin/kubectl
chmod +x /usr/local/bin/kubectl
fi
# 部署集群
runuser -l $rke_user -c "rke up --config /home/$rke_user/cluster.yml"
export KUBECONFIG=/home/$rke_user/kube_config_cluster.yml
echo "cluster nodes:"
kubectl get no
使用方法,登录其中一个节点。
chmod +x ./main.sh
# ip.txt在同一个目录下
./main.sh
然后等待, 结果:
脚本缺点:
上面的脚本只是简单实现了功能, 并没有深入去写,存在一些问题。 比如:
- 异常处理并没有怎么做。 如果更进一步改进,可以加ip.txt文件的检查,格式检查,docker等软件是否安装的检查,环境配置检查等。
- 脚本安装rke的下载链接并不稳定,我没找到国内的rke安装包镜像, 可能下载的很慢。 没有下载成功需要手动下载安装rke。