一、环境
角色 | IP |
---|---|
etcd-1 | 192.168.10.10 (12379、12380) |
etcd-2 | 192.168.10.10 (22379、22380) |
etcd-3 | 192.168.10.10 (32379、32380) |
多服务器把ip和端口自行调整一下就行了
二、软件下载
mkdir /opt/etcd
mkdir /opt/etcd/{cfg,ssl,data,wal} –p
mkdir /opt/etcd2
mkdir /opt/etcd2/{cfg,ssl,data,wal} –p
mkdir /opt/etcd3
mkdir /opt/etcd3/{cfg,ssl,data,wal} –p
工具下载:
curl -L https://github.com/cloudflare/cfssl/releases/download/v1.5.0/cfssl_1.5.0_linux_amd64 -o cfssl
chmod +x cfssl
curl -L https://github.com/cloudflare/cfssl/releases/download/v1.5.0/cfssljson_1.5.0_linux_amd64 -o cfssljson
chmod +x cfssljson
curl -L https://github.com/cloudflare/cfssl/releases/download/v1.5.0/cfssl-certinfo_1.5.0_linux_amd64 -o cfssl-certinfo
chmod +x cfssl-certinfo
mv {cfssl,cfssljson,cfssl-certinfo} /usr/local/bin
etcd软件下载:
wget -c https://github.com/etcd-io/etcd/releases/download/v3.5.0/etcd-v3.5.0-linux-amd64.tar.gz
tar -zxf etcd-v3.5.0-linux-amd64.tar.gz
cd etcd-v3.5.0-linux-amd64
mv {etcd,etcdctl,etcdutl} /usr/local/bin/
三、证书生成
(1)自签证书颁发机构(CA)
[root@localhost etcd]# cat > ca-config.json<< EOF
{
"signing":{
"default":{
"expiry":"87600h"
},
"profiles":{
"kubernetes":{
"expiry":"87600h",
"usages":[
"signing",
"key encipherment",
"server auth",
"client auth"
]
}
}
}
}
EOF
[root@localhost etcd]# cat > ca-csr.json<< EOF
{
"CN":"etcd CA",
"key":{
"algo":"rsa",
"size":2048
},
"names":[
{
"C":"CN",
"L":"Beijing",
"ST":"Beijing"
}
]
}
EOF
生成 CA 秘钥文件(
ca-key.pem
)和证书文件(ca.pem
) :
[root@localhost etcd]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca
2022/07/31 17:15:14 [INFO] generating a new CA key and certificate from CSR
2022/07/31 17:15:14 [INFO] generate received request
2022/07/31 17:15:14 [INFO] received CSR
2022/07/31 17:15:14 [INFO] generating key: rsa-2048
2022/07/31 17:15:15 [INFO] encoded CSR
2022/07/31 17:15:15 [INFO] signed certificate with serial number 504349668567155459345189436720647214038928670128
(2)使用自签 CA 签发 Etcd HTTPS 证书
创建证书申请文件:
cat > etcd-csr.json<< EOF
{
"CN":"etcd",
"hosts":[
"192.168.66.31",
"192.168.66.41",
"192.168.66.42"
],
"key":{
"algo":"rsa",
"size":2048
},
"names":[
{
"C":"CN",
"L":"BeiJing",
"ST":"BeiJing"
}
]
}
EOF
生成证书: 为 API 服务器生成秘钥和证书,默认会分别存储为
etcd-key.pem
和 etcd.pem
两个文件。
[root@localhost etcd]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes etcd-csr.json | cfssljson -bare etcd
2022/07/31 17:34:11 [INFO] generate received request
2022/07/31 17:34:11 [INFO] received CSR
2022/07/31 17:34:11 [INFO] generating key: rsa-2048
2022/07/31 17:34:11 [INFO] encoded CSR
2022/07/31 17:34:11 [INFO] signed certificate with serial number 550588339086205748107774212753833209082394411557
为etcd放置证书
mv {ca.pem , etcd-key.pem , etcd.pem} /opt/etcd/ssl
cp -r /opt/etcd/ssl /opt/etcd2
cp -r /opt/etcd/ssl /opt/etcd3
四、etcd配置文件
命令行参数:Configuration flags | etcd
yaml配置文件:etcd/etcd.conf.yml.sample at main · etcd-io/etcd · GitHub
这里先在etcd-1配置好,下面的etcd-2、etcd-3等后面再配置
etcd-1:/opt/etcd/cfg/etcd.yml
name: "etcd-1"
data-dir: "/opt/etcd/data"
wal-dir: "/opt/etcd/wal"
# 用于侦听对等流量的逗号分隔的url列表。
listen-peer-urls: https://192.168.10.10:12380
# 用于侦听客户机通信的逗号分隔的url列表。
listen-client-urls: https://192.168.10.10:12379
# 这个成员的对等url的列表,以通告给集群的其他成员。url需要是逗号分隔的列表。
initial-advertise-peer-urls: https://192.168.10.10:12380
#这个成员的对等url的列表,以通告给集群的其他成员。url需要是逗号分隔的列表。
advertise-client-urls: https://192.168.10.10:12379
# Initial cluster configuration for bootstrapping.
initial-cluster: 'etcd-1=https://192.168.10.10:12380,etcd-2=https://192.168.10.10:22380,etcd-3=https://192.168.10.10:32380'
# Initial cluster token for the etcd cluster during bootstrap.
initial-cluster-token: 'etcd-cluster'
# Initial cluster state ('new' or 'existing').
initial-cluster-state: 'new'
client-transport-security:
# Path to the client server TLS cert file.
cert-file: /opt/etcd/ssl/etcd.pem
# Path to the client server TLS key file.
key-file: /opt/etcd/ssl/etcd-key.pem
# Path to the client server TLS trusted CA cert file.
trusted-ca-file: /opt/etcd/ssl/ca.pem
peer-transport-security:
# Path to the peer server TLS cert file.
cert-file: /opt/etcd/ssl/etcd.pem
# Path to the peer server TLS key file.
key-file: /opt/etcd/ssl/etcd-key.pem
# Path to the peer server TLS trusted CA cert file.
trusted-ca-file: /opt/etcd/ssl/ca.pem
etcd-2:/opt/etcd2/cfg/etcd.yml
name: "etcd-2"
data-dir: "/opt/etcd2/data"
wal-dir: "/opt/etcd2/wal"
# 用于侦听对等流量的逗号分隔的url列表。
listen-peer-urls: https://192.168.10.10:22380
# 用于侦听客户机通信的逗号分隔的url列表。
listen-client-urls: https://192.168.10.10:22379
# 这个成员的对等url的列表,以通告给集群的其他成员。url需要是逗号分隔的列表。
initial-advertise-peer-urls: https://192.168.10.10:22380
#这个成员的对等url的列表,以通告给集群的其他成员。url需要是逗号分隔的列表。
advertise-client-urls: https://192.168.10.10:22379
# Initial cluster configuration for bootstrapping.
initial-cluster: 'etcd-1=https://192.168.10.10:12380,etcd-2=https://192.168.10.10:22380,etcd-3=https://192.168.10.10:32380'
# Initial cluster token for the etcd2 cluster during bootstrap.
initial-cluster-token: 'etcd-cluster'
# Initial cluster state ('new' or 'existing').
initial-cluster-state: 'new'
client-transport-security:
# Path to the client server TLS cert file.
cert-file: /opt/etcd2/ssl/etcd.pem
# Path to the client server TLS key file.
key-file: /opt/etcd2/ssl/etcd-key.pem
# Path to the client server TLS trusted CA cert file.
trusted-ca-file: /opt/etcd2/ssl/ca.pem
peer-transport-security:
# Path to the peer server TLS cert file.
cert-file: /opt/etcd2/ssl/etcd.pem
# Path to the peer server TLS key file.
key-file: /opt/etcd2/ssl/etcd-key.pem
# Path to the peer server TLS trusted CA cert file.
trusted-ca-file: /opt/etcd2/ssl/ca.pem
etcd-3
name: "etcd-3"
data-dir: "/opt/etcd3/data"
wal-dir: "/opt/etcd3/wal"
# 用于侦听对等流量的逗号分隔的url列表。
listen-peer-urls: https://192.168.10.10:32380
# 用于侦听客户机通信的逗号分隔的url列表。
listen-client-urls: https://192.168.10.10:32379
# 这个成员的对等url的列表,以通告给集群的其他成员。url需要是逗号分隔的列表。
initial-advertise-peer-urls: https://192.168.10.10:32380
#这个成员的对等url的列表,以通告给集群的其他成员。url需要是逗号分隔的列表。
advertise-client-urls: https://192.168.10.10:32379
# Initial cluster configuration for bootstrapping.
initial-cluster: 'etcd-1=https://192.168.10.10:12380,etcd-2=https://192.168.10.10:22380,etcd-3=https://192.168.10.10:32380'
# Initial cluster token for the etcd cluster during bootstrap.
initial-cluster-token: 'etcd-cluster'
# Initial cluster state ('new' or 'existing').
initial-cluster-state: 'new'
client-transport-security:
# Path to the client server TLS cert file.
cert-file: /opt/etcd3/ssl/etcd.pem
# Path to the client server TLS key file.
key-file: /opt/etcd3/ssl/etcd-key.pem
# Path to the client server TLS trusted CA cert file.
trusted-ca-file: /opt/etcd3/ssl/ca.pem
peer-transport-security:
# Path to the peer server TLS cert file.
cert-file: /opt/etcd3/ssl/etcd.pem
# Path to the peer server TLS key file.
key-file: /opt/etcd3/ssl/etcd-key.pem
# Path to the peer server TLS trusted CA cert file.
trusted-ca-file: /opt/etcd3/ssl/ca.pem
五 服务配置
etcd.service
cat > /usr/lib/systemd/system/etcd.service << EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
ExecStart=/usr/local/bin/etcd --config-file /opt/etcd/cfg/etcd.yml
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF
etcd2.service
cat > /usr/lib/systemd/system/etcd2.service << EOF
[Unit]
Description=Etcd2 Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
ExecStart=/usr/local/bin/etcd --config-file /opt/etcd2/cfg/etcd.yml
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF
etcd3.service
cat > /usr/lib/systemd/system/etcd3.service << EOF
[Unit]
Description=Etcd3 Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
ExecStart=/usr/local/bin/etcd --config-file /opt/etcd3/cfg/etcd.yml
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF
六、启动服务
systemctl enable etcd --now
systemctl enable etcd2 --now
systemctl enable etcd3 --now
七、集群状态查看
[root@k8s-master01 ~]# etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/etcd.pem --key=/opt/etcd/ssl/etcd-key.pem --endpoints="https://192.168.10.10:12379,https://192.168.10.10:22379,https://192.168.10.10:32379" endpoint status --write-out=table
+-----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+-----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| https://192.168.10.10:12379 | 3910c98017de91db | 3.5.0 | 2.4 MB | true | false | 4 | 2114 | 2114 | |
| https://192.168.10.10:22379 | 6c512aefe1e16758 | 3.5.0 | 2.4 MB | false | false | 4 | 2114 | 2114 | |
| https://192.168.10.10:32379 | 4f6354e52760b515 | 3.5.0 | 2.4 MB | false | false | 4 | 2114 | 2114 | |
+-----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
八、文件生成完毕,查看结果
- cfg 步骤四生成的etcd yaml配置文件 ,etcd.yml
- data etcd数据文件目录
- wal 日志文件目录
- ssl 证书文件,ca.pem server-key.pem server.pem