04-高可用Kubernetes集群(测试环境)-部署Etcd集群

1. 安装前准备

关闭各节点的selinux，设置SELINUX=disabled，重启节点。各节点关闭并禁止开机启动防火墙.

# 关闭SELINUX
vi /etc/selinux/config  
reboot
setenforce 0

# 关闭防火墙并禁止开机启动
systemctl stop firewalld
systemctl disable firewalld

2. 下载etcd

etcd二进制包下载地址：https://github.com/etcd-io/etcd/releases，这里使用 etcd-v3.3.10-linux-amd64.tar.gz，下载后解压缩。

[root@master01 k8s]# tar zxf etcd-v3.3.10-linux-amd64.tar.gz

3. 安装etcd

创建安装目录, 将etcdctl etcd拷贝至对应目录,命令如下:

mkdir /opt/etcd/{bin,cfg,ssl} -p
cd etcd-v3.3.10-linux-amd64/
mv etcdctl etcd /opt/etcd/bin

拷贝证书到指定目录， 执行安装脚本etcd.sh

cd k8s
chmod +x etcd.sh

cp etcd-cert/ca*pem /opt/etcd/ssl/
cp etcd-cert/server*pem /opt/etcd/ssl/
./etcd.sh etcd01 192.168.1.72 etcd02=https://192.168.1.95:2380,etcd03=https://192.168.1.88:2380

etcd.sh

#!/bin/bash
# example: ./etcd.sh etcd01 192.168.1.10 etcd02=https://192.168.1.11:2380,etcd03=https://192.168.1.12:2380

ETCD_NAME=$1
ETCD_IP=$2
ETCD_CLUSTER=$3

WORK_DIR=/opt/etcd

cat <<EOF >$WORK_DIR/cfg/etcd
#[Member]
ETCD_NAME="${ETCD_NAME}"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://${ETCD_IP}:2380"
ETCD_LISTEN_CLIENT_URLS="https://${ETCD_IP}:2379"

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://${ETCD_IP}:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://${ETCD_IP}:2379"
ETCD_INITIAL_CLUSTER="etcd01=https://${ETCD_IP}:2380,${ETCD_CLUSTER}"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF

cat <<EOF >/usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=${WORK_DIR}/cfg/etcd
ExecStart=${WORK_DIR}/bin/etcd \
--name=\${ETCD_NAME} \
--data-dir=\${ETCD_DATA_DIR} \
--listen-peer-urls=\${ETCD_LISTEN_PEER_URLS} \
--listen-client-urls=\${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 \
--advertise-client-urls=\${ETCD_ADVERTISE_CLIENT_URLS} \
--initial-advertise-peer-urls=\${ETCD_INITIAL_ADVERTISE_PEER_URLS} \
--initial-cluster=\${ETCD_INITIAL_CLUSTER} \
--initial-cluster-token=\${ETCD_INITIAL_CLUSTER_TOKEN} \
--initial-cluster-state=new \
--cert-file=${WORK_DIR}/ssl/server.pem \
--key-file=${WORK_DIR}/ssl/server-key.pem \
--peer-cert-file=${WORK_DIR}/ssl/server.pem \
--peer-key-file=${WORK_DIR}/ssl/server-key.pem \
--trusted-ca-file=${WORK_DIR}/ssl/ca.pem \
--peer-trusted-ca-file=${WORK_DIR}/ssl/ca.pem
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable etcd
systemctl restart etcd

拷贝etcd相关文件到其他两个etcd节点(node01和node02)

scp -r /opt/etcd/ root@192.168.1.95:/opt
scp -r /opt/etcd/ root@192.168.1.88:/opt
scp /usr/lib/systemd/system/etcd.service root@192.168.1.95:/usr/lib/systemd/system/
scp /usr/lib/systemd/system/etcd.service root@192.168.1.88:/usr/lib/systemd/system/

修改node01和node02节点上的/opt/etcd/cfg/etcd，对应的名称和IP，改为对应名称和本机IP。其中node01节点上修改如下，node02节点做类似操作。

vi /opt/etcd/cfg/etcd

启动etcd, 并将etcd设置为开机自启动

systemctl daemon-reload
systemctl enable etcd
systemctl start etcd

4. 检查集群

在各节点均可检查,

# 进入证书目录
cd /opt/etcd/ssl

# 集群健康情况
/opt/etcd/bin/etcdctl \
--ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem \
--endpoints="https://192.168.1.72:2379,https://192.168.1.95:2379,https://192.168.1.88:2379" \
cluster-health

# 集群成员
/opt/etcd/bin/etcdctl \
--ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem \
--endpoints="https://192.168.1.72:2379,https://192.168.1.95:2379,https://192.168.1.88:2379" \
member list

集群各节点状态为healthy, leader 为etcd01. 至此, etcd三节点集群部署成功.