k8s之etcd集群部署

下载安装包（etcd最新版本3.4.0）

wget https://github.com/etcd-io/etcd/releases/download/v3.4.0/etcd-v3.4.0-linux-amd64.tar.gz
tar -xvf etcd-v3.3.10-linux-amd64.tar.gz
cp etcd etcdctl /k8s/etcd/bin/

修改配置文件

cat << EOF > /k8s/etcd/cfg/etcd
\#[Member]
ETCD_NAME="etcd01"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.0.1:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.0.1:2379"

\#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.0.1:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.0.1:2379"
ETCD_INITIAL_CLUSTER="etcd01=https://192.168.0.1:2380,etcd02=https://192.168.0.2:2380,etcd03=https://192.168.0.3:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_ENABLE_V2="true"
EOF

提示：

1. 其他etcd节点按照以上配置文件修改其中红字部分即可
2. flannel操作etcd使用的是v2的API，而kubernetes操作etcd使用的v3的API，为了兼容flannel，将默认开启v2版本，故配置文件中设置 ETCD_ENABLE_V2=“true”

创建TLS 密钥和证书

为了保证通信安全，客户端(如etcdctl)与etcd 集群、etcd 集群之间的通信需要使用TLS 加密。
创建etcd 证书签名请求：

cat > etcd-csr.json <<EOF
{
 "CN": "etcd",
 "hosts": [
   "192.168.0.1",
   "192.168.0.2",
   "192.168.0.3"
 ],
 "key": {
   "algo": "rsa",
   "size": 2048
 },
 "names": [
   {
    "C": "CN",
    "ST": "BeiJing",
    "L": "BeiJing",
    "O": "k8s",
    "OU": "System"
   }
 ]
}
EOF

生成etcd证书和私钥

\# cfssl gencert -ca=/k8s/kubernetes/ssl/ca.pem -ca-key=/k8s/kubernetes/ssl/ca-key.pem  -config=/k8s/kubernetes/ssl/ca-config.json  -profile=kubernetes etcd-csr.json | cfssljson -bare etcd

\# ls etcd*
etcd.csr  etcd-csr.json  etcd-key.pem  etcd.pem

创建 etcd的 systemd unit 文件

cat << EOF > /lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=/k8s/etcd/cfg/etcd
ExecStart=/k8s/etcd/bin/etcd \
--cert-file=/k8s/etcd/ssl/etcd.pem \
--key-file=/k8s/etcd/ssl/etcd-key.pem \
--peer-cert-file=/k8s/etcd/ssl/etcd.pem \
--peer-key-file=/k8s/etcd/ssl/etcd-key.pem \
--trusted-ca-file=/k8s/kubernetes/ssl/ca.pem \
--peer-trusted-ca-file=/k8s/kubernetes/ssl/ca.pem
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

分发文件至其他节点

cd /k8s/ 
scp -r etcd/ 192.168.0.2:/k8s/
scp -r etcd/ 192.168.0.3:/k8s/

scp /lib/systemd/system/etcd.service 192.168.0.2:/lib/systemd/system/etcd.service
scp /lib/systemd/system/etcd.service 192.168.0.3:/lib/systemd/system/etcd.service

启动etcd 服务

> systemctl daemon-reload
> systemctl enable etcd
> systemctl start etcd

> \# /k8s/etcd/bin/etcdctl --cacert=/k8s/kubernetes/ssl/ca.pem --cert=/k8s/etcd/ssl/etcd.pem --key=/k8s/etcd/ssl/etcd-key.pem --endpoints="https://192.168.0.3:2379,https://192.168.0.2:2379,https://192.168.0.1:2379" endpoint health
> https://192.168.0.2:2379 is healthy: successfully committed proposal: took = 24.271259ms
> https://192.168.0.3:2379 is healthy: successfully committed proposal: took = 31.633027ms
> https://192.168.0.1:2379 is healthy: successfully committed proposal: took = 37.463262ms

etcd 3.4注意事项

• - ETCD3.4版本ETCDCTL_API=3 etcdctl 和 etcd --enable-v2=false 成为了默认配置，如要使用v2版本，执行etcdctl时候需要设置ETCDCTL_API环境变量，例如：ETCDCTL_API=2 etcdctl
  
  

• - ETCD3.4版本会自动读取环境变量的参数，所以EnvironmentFile文件中有的参数，不需要再次在ExecStart启动参数中添加，二选一，如同时配置，会触发以下类似报错“etcd: conflicting environment variable "ETCD_NAME" is shadowed by corresponding command-line flag (either unset environment variable or disable flag)”
  
  

• - flannel操作etcd使用的是v2的API，而kubernetes操作etcd使用的v3的API