etcd作为k8s的后端存储,重要性不言而喻。etcd集群挂掉对k8s集群会造成灾难性后果。因此也是我们运维的时候重要的关注点。如果规模比较大还是建议使用SSD作为数据盘来提高速度。
1 下载etcd安装包
wget https://github.com/coreos/etcd/releases/download/v3.3.7/etcd-v3.3.7-linux-amd64.tar.gz
2 解压安装etcd(3台主机做同样配置)
mkdir /var/lib/etcd
tar -zxvf etcd-v3.3.7-linux-amd64.tar.gz
cp etcd etcdctl /opt/kubernetes/bin
4创建etcd启动文件
cat > /usr/lib/systemd/system/etcd.service <<EOF
[Unit]
Description=Etcd Server
After=network.target
[Service]
Type=simple
WorkingDirectory=/var/lib/etcd
EnvironmentFile=/opt/kubernetes/conf/etcd.conf
ExecStart=/bin/bash -c "GOMAXPROCS=1 /opt/kubernetes/bin/etcd"
Type=notify
[Install]
WantedBy=multi-user.target
EOF
5 编译etcd.conf文件
vi /opt/kubernetes/conf/ectd.conf
#[Member]
#ETCD_CORS=""
ETCD_DATA_DIR="/var/lib/etcd/local.etcd"
#ETCD_WAL_DIR=""
ETCD_LISTEN_PEER_URLS="https://0.0.0.0:2380"
#ETCD_LISTEN_CLIENT_URLS="https://0.0.0.0:2379,http://127.0.0.1:4001"
ETCD_LISTEN_CLIENT_URLS="https://0.0.0.0:2379"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
ETCD_NAME="host-10-20-23-155"
#ETCD_SNAPSHOT_COUNT="100000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
#ETCD_QUOTA_BACKEND_BYTES="0"
#ETCD_MAX_REQUEST_BYTES="1572864"
#ETCD_GRPC_KEEPALIVE_MIN_TIME="5s"
#ETCD_GRPC_KEEPALIVE_INTERVAL="2h0m0s"
#ETCD_GRPC_KEEPALIVE_TIMEOUT="20s"
#
#[Clustering]
#ETCD_INITIAL_ADVERTISE_PEER_URLS="http://localhost:2380"
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://host-10-20-23-155:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://host-10-20-23-155:2379"
#ETCD_DISCOVERY=""
#ETCD_DISCOVERY_FALLBACK="proxy"
#ETCD_DISCOVERY_PROXY=""
#ETCD_DISCOVERY_SRV=""
ETCD_INITIAL_CLUSTER="host-10-20-23-155=https://host-10-20-23-155:2380,host-10-20-23-53=https://host-10-20-23-53:2380,host-10-20-23-200=https://host-10-20-23-200:23
80"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
#ETCD_STRICT_RECONFIG_CHECK="true"
#ETCD_ENABLE_V2="true"
#
#[Proxy]
#ETCD_PROXY="off"
#ETCD_PROXY_FAILURE_WAIT="5000"
#ETCD_PROXY_REFRESH_INTERVAL="30000"
#ETCD_PROXY_DIAL_TIMEOUT="1000"
#ETCD_PROXY_WRITE_TIMEOUT="5000"
#ETCD_PROXY_READ_TIMEOUT="0"
#
#[Security]
CLIENT_CERT_AUTH="true"
ETCD_CA_FILE="/opt/kubernetes/ssl/ca.pem"
ETCD_CERT_FILE="/opt/kubernetes/ssl/etcd.pem"
6 各etcd节点启动etcd并设置开机自动启动
#systemctl daemon-reload
#systemctl enable etcd
#systemctl start etcd.service
#systemctl status etcd.service
6 分发各etcd节点测试验证etcd集群配置
#etcd --version //查看etcd安装版本
etcd Version: 3.3.7
Git SHA: 56536de55
Go Version: go1.9.6
Go OS/Arch: linux/amd64
查看etcd健康集群状态
etcdctl --endpoints=https://etcd01:2379 \
--ca-file=/opt/kubernetes/ssl/ca.pem \
--cert-file=/opt/kubernetes/ssl/etcd.pem \
--key-file=/opt/kubernetes/ssl/etcd-key.pem cluster-health
member b1840b0a404e1103 is healthy: got healthy result from
https://10.20.24.15:2379 member d15b66900329a12d is healthy: got
healthy result from https://hadoop3:2379 member f9794412c46a9cb0 is
healthy: got healthy result from https://hadoop2:2379 cluster is
healthy
PS:
- systemd启动etcd服务的时候出现错误:Failed at step CHDIR spawning /usr/bin/etcd: No such file or directory
解决办法:etcd.service服务配置文件中设置的工作目录WorkingDirectory=/var/lib/etcd/必须存在,否则会报以上错误 - etcd集群启动之后最好把下面的更改
ETCD_INITIAL_CLUSTER_STATE=“new”
改为
ETCD_INITIAL_CLUSTER_STATE=“existing”
否则如果集群全部停机会出现无法启动的现象