k8s二进制安装篇2-部署etcd集群

etcd 的leader选举机制，要求至少为3台或以上的奇数台。本次安装涉及：192.168.1.246，192.168.1.247，192.168.1.248

一，签发etcd证书

证书签发服务器192.168.1.245:
1，创建ca的json配置: /opt/certs/ca-config.json
oserver 表示服务端连接客户端时携带的证书，用于客户端验证服务端身份
oclient 表示客户端连接服务端时携带的证书，用于服务端验证客户端身份
opeer 表示相互之间连接时使用的证书，如etcd节点之间验证
“expiry”: “175200h” 证书有效期 十年 如果这里是一年的话 到期后集群会立宕掉

vim  /opt/certs/ca-config.json
{
    "signing": {
        "default": {
            "expiry": "175200h"
        },
        "profiles": {
            "server": {
                "expiry": "175200h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "server auth"
                ]
            },
            "client": {
                "expiry": "175200h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "client auth"
                ]
            },
            "peer": {
                "expiry": "175200h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "server auth",
                    "client auth"
                ]
            }
        }
    }
}

2，创建etcd证书配置：/opt/certs/etcd-peer-csr.json
重点在hosts上，将所有可能的etcd服务器添加到host列表，不能使用网段，新增etcd服务器需要重新签发证书

vim /opt/certs/etcd-peer-csr.json
{
    "CN": "k8s-etcd",
    "hosts": [
        "192.168.1.245",
        "192.168.1.246",
        "192.168.1.247",
        "192.168.1.248"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "ST": "guangdong",
            "L": "guangzhou",
            "O": "zz",
            "OU": "ops"
        }
    ]
}

3，签发证书

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=peer etcd-peer-csr.json |cfssljson -bare etcd-peer

二，安装etcd

etcd地址：https://github.com/etcd-io/etcd/
下载地址：https://github.com/etcd-io/etcd/releases/download/v3.4.0/etcd-v3.4.0-linux-amd64.tar.gz
本次安装涉及：192.168.1.246，192.168.1.247，192.168.1.248
1，下载etcd（192.168.1.246，192.168.1.247，192.168.1.248）

cd /usr/local/src/
wget https://github.com/etcd-io/etcd/releases/download/v3.4.0/etcd-v3.4.0-linux-amd64.tar.gz
useradd -s /sbin/nologin -M etcd
tar -xf etcd-v3.4.0-linux-amd64.tar.gz 
mv etcd-v3.4.0-linux-amd64 /opt/etcd-v3.4.0
cd /opt/etcd-v3.4.0
ln -s /opt/etcd-v3.4.0/  /opt/etcd
mkdir -p /opt/etcd/certs  /data/etcd  /data/logs/etcd-server

2，下发证书到各个etcd上（192.168.1.245）

cd /opt/certs/
for i in 246 247 248;do scp ca.pem etcd-peer.pem etcd-peer-key.pem 192.168.1.${i}:/opt/etcd/certs/ ;done

3，创建启动脚本(部分参数每台机器不同)（192.168.1.246，192.168.1.247，192.168.1.248）

vim /lib/systemd/system/etcd.service
[Unit]
Description=etcd
Documentation=https://github.com/coreos

[Service]
Type=notify
ExecStart=/opt/etcd/etcd \
--name=etcd-server-246 \
--data-dir=/data/etcd/etcd-server \
--listen-peer-urls https://192.168.1.246:2380 \
--listen-client-urls https://192.168.1.246:2379,http://127.0.0.1:2379 \
--quota-backend-bytes 8000000000 \
--initial-advertise-peer-urls https://192.168.1.246:2380 \
--advertise-client-urls https://192.168.1.246:2379,http://127.0.0.1:2379 \
--initial-cluster  etcd-server-246=https://192.168.1.246:2380,etcd-server-247=https://192.168.1.247:2380,etcd-server-248=https://192.168.1.248:2380 \
--initial-cluster-token etcd-cluster-0 \
--cert-file=/opt/etcd/certs/etcd-peer.pem \
--key-file=/opt/etcd/certs/etcd-peer-key.pem \
--client-cert-auth  \
--peer-client-cert-auth \
--trusted-ca-file=/opt/etcd/certs/ca.pem \
--peer-cert-file=/opt/etcd/certs/etcd-peer.pem \
--peer-key-file=/opt/etcd/certs/etcd-peer-key.pem \
--peer-client-cert-auth \
--peer-trusted-ca-file=/opt/etcd/certs/ca.pem \
--enable-v2 \ # 由于flannel目前确实不能与etcdV3直接交互，开启etcd 支持V2api功能，在etcd启动参数中加入 --enable-v2参数

Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

启动etcd

systemctl daemon-reload
systemctl start etcd.service
systemctl enable etcd.service

查看集群的状态
./etcdctl   --cacert=/opt/etcd/certs/ca.pem --cert=/opt/etcd/certs/etcd-peer.pem   --key=/opt/etcd/certs/etcd-peer-key.pem  member list --endpoints=https://192.168.1.246:2379,https://192.168.1.247:2379,https://192.168.1.248:2379

查看leader
etcd-v3.4.0]# curl http://127.0.0.1:2379/v2/stats/leader
{"leader":"e87617ab250d1acb","followers":{"7e037e71de645e58":{"latency":{"current":0.002364,"average":0.005725999999999999,"standardDeviation":0.0031717232434960887,"minimum":0.002347,"maximum":0.010384},"counts":{"fail":0,"success":6}},"dae8c96b50d21d09":{"latency":{"current":0.002043,"average":0.0039885,"standardDeviation":0.0016390976938547633,"minimum":0.002043,"maximum":0.00654},"counts":{"fail":0,"success":6}}}}


etcd-v3.4.0]# curl http://127.0.0.1:2379/v2/stats/leader
{"message":"not current leader"}