一、环境准备:
10.10.0.170 k8s-master
10.10.0.171 k8s-node1
10.10.0.172 k8s-node2
二、安装:
2.1 建立主机信任:
k8s-master上执行下列命令:
ssh-keygen -t rsa #一路回车即可
ssh-copy-id k8s-master
ssh-copy-id k8s-node1
ssh-copy-id k8s-node2
2.2 设置cfssl环境(k8s-master上执行):
wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
chmod +x cfssl_linux-amd64
mv cfssl_linux-amd64 /usr/local/bin/cfssl
chmod +x cfssljson_linux-amd64
mv cfssljson_linux-amd64 /usr/local/bin/cfssljson
chmod +x cfssl-certinfo_linux-amd64
mv cfssl-certinfo_linux-amd64 /usr/local/bin/cfssl-certinfo
2.3 创建CA配置文件:
cat > ca-config.json <<EOF { "signing": { "default": { "expiry": "8760h" }, "profiles": { "kubernetes-Soulmate": { "usages": [ "signing", "key encipherment", "server auth", "client auth" ], "expiry": "8760h" } } } } EOF cat > ca-csr.json <<EOF { "CN": "kubernetes-Soulmate", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "shanghai", "L": "shanghai", "O": "k8s", "OU": "System" } ] } EOF cfssl gencert -initca ca-csr.json | cfssljson -bare ca cat > etcd-csr.json <<EOF { "CN": "etcd", "hosts": [ "127.0.0.1", "10.10.0.170", "10.10.0.171", "10.10.0.172" ], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "shanghai", "L": "shanghai", "O": "k8s", "OU": "System" } ] } EOF cfssl gencert -ca=ca.pem \ -ca-key=ca-key.pem \ -config=ca-config.json \ -profile=kubernetes-Soulmate etcd-csr.json | cfssljson -bare etcd
[root@k8s-master ssl]# ls
ca-config.json
ca.csr
ca-csr.json
ca-key.pem
ca.pem
etcd.csr
etcd-csr.json
etcd-key.pem
etcd.pem
2.4 将etcd的证书分发到k8s-node1、 k8s-node2(k8s-master上执行):
mkdir /etc/etcd/ssl/
cp etcd.pem etcd-key.pem ca.pem /etc/etcd/ssl/ ssh -n k8s-node1 "mkdir -p /etc/etcd/ssl && exit" ssh -n k8s-node2 "mkdir -p /etc/etcd/ssl && exit" scp -r /etc/etcd/ssl/*.pem k8s-node1:/etc/etcd/ssl/ scp -r /etc/etcd/ssl/*.pem k8s-node2:/etc/etcd/ssl/
三、安装etcd(3个节点都执行):
yum install etcd -y
四、etcd.service配置:
k8s-master:
[root@k8s-master ssl]# cat /etc/systemd/system/etcd.service [Unit] Description=Etcd Server After=network.target After=network-online.target Wants=network-online.target Documentation=https://github.com/coreos [Service] Type=notify WorkingDirectory=/var/lib/etcd/ ExecStart=/usr/bin/etcd --name k8s-master --cert-file=/etc/etcd/ssl/etcd.pem --key-file=/etc/etcd/ssl/etcd-key.pem --peer-cert-file=/etc/etcd/ssl/etcd.pem --peer-key-file=/etc/etcd/ssl/etcd-key.pem --trusted-ca-file=/etc/etcd/ssl/ca.pem --peer-trusted-ca-file=/etc/etcd/ssl/ca.pem --initial-advertise-peer-urls https://10.10.0.170:2380 --listen-peer-urls https://10.10.0.170:2380 --listen-client-urls https://10.10.0.170:2379,http://127.0.0.1:2379 --advertise-client-urls https://10.10.0.170:2379 --initial-cluster-token etcd-cluster-0 --initial-cluster k8s-master=https://10.10.0.170:2380,k8s-node1=https://10.10.0.171:2380,k8s-node2=https://10.10.0.172:2380 --initial-cluster-state new --data-dir=/var/lib/etcd Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target
k8s-node1:
[root@k8s-node1 ~]# cat /etc/systemd/system/etcd.service [Unit] Description=Etcd Server After=network.target After=network-online.target Wants=network-online.target Documentation=https://github.com/coreos [Service] Type=notify WorkingDirectory=/var/lib/etcd/ ExecStart=/usr/bin/etcd --name k8s-node1 --cert-file=/etc/etcd/ssl/etcd.pem --key-file=/etc/etcd/ssl/etcd-key.pem --peer-cert-file=/etc/etcd/ssl/etcd.pem --peer-key-file=/etc/etcd/ssl/etcd-key.pem --trusted-ca-file=/etc/etcd/ssl/ca.pem --peer-trusted-ca-file=/etc/etcd/ssl/ca.pem --initial-advertise-peer-urls https://10.10.0.171:2380 --listen-peer-urls https://10.10.0.171:2380 --listen-client-urls https://10.10.0.171:2379,http://127.0.0.1:2379 --advertise-client-urls https://10.10.0.171:2379 --initial-cluster-token etcd-cluster-0 --initial-cluster k8s-master=https://10.10.0.170:2380,k8s-node1=https://10.10.0.171:2380,k8s-node2=https://10.10.0.172:2380 --initial-cluster-state new --data-dir=/var/lib/etcd Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target
k8s-node2:
[root@k8s-node2 ~]# cat /etc/systemd/system/etcd.service [Unit] Description=Etcd Server After=network.target After=network-online.target Wants=network-online.target Documentation=https://github.com/coreos [Service] Type=notify WorkingDirectory=/var/lib/etcd/ ExecStart=/usr/bin/etcd --name k8s-node2 --cert-file=/etc/etcd/ssl/etcd.pem --key-file=/etc/etcd/ssl/etcd-key.pem --peer-cert-file=/etc/etcd/ssl/etcd.pem --peer-key-file=/etc/etcd/ssl/etcd-key.pem --trusted-ca-file=/etc/etcd/ssl/ca.pem --peer-trusted-ca-file=/etc/etcd/ssl/ca.pem --initial-advertise-peer-urls https://10.10.0.172:2380 --listen-peer-urls https://10.10.0.172:2380 --listen-client-urls https://10.10.0.172:2379,http://127.0.0.1:2379 --advertise-client-urls https://10.10.0.172:2379 --initial-cluster-token etcd-cluster-0 --initial-cluster k8s-master=https://10.10.0.170:2380,k8s-node1=https://10.10.0.171:2380,k8s-node2=https://10.10.0.172:2380 --initial-cluster-state new --data-dir=/var/lib/etcd Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target
参数解释:
--listen-peer-urls
监听的用于节点之间通信的url,可监听多个,集群内部将通过这些url进行数据交互(如选举,数据同步等)
--initial-advertise-peer-urls
建议用于节点之间通信的url,节点间将以该值进行通信。
--listen-client-urls
监听的用于客户端通信的url,同样可以监听多个。
--advertise-client-urls
建议使用的客户端通信url,该值用于etcd代理或etcd成员与etcd节点通信。
--initial-cluster-token etcd-cluster-1
节点的token值,设置该值后集群将生成唯一id,并为每个节点也生成唯一id,当使用相同配置文件再启动一个集群时,只要该token值不一样,etcd集群就不会相互影响。
--initial-cluster
也就是集群中所有的initial-advertise-peer-urls 的合集
--initial-cluster-state new
新建集群的标志
三个节点执行下列命令:
systemctl daemon-reload
systemctl enable etcd
systemctl start etcd
systemctl status etcd
检查etcd集群健康性(可三个节点都试试):
[root@k8s-master ssl]# etcdctl --endpoints=https://10.10.0.170:2379,https://10.10.0.171:2379,https://10.10.0.172:2379 \ > --ca-file=/etc/etcd/ssl/ca.pem \ > --cert-file=/etc/etcd/ssl/etcd.pem \ > --key-file=/etc/etcd/ssl/etcd-key.pem cluster-health member 1c25bde2973f71cf is healthy: got healthy result from https://10.10.0.172:2379 member 3222a6aebdf856ac is healthy: got healthy result from https://10.10.0.170:2379 member 5796b25a0b404b92 is healthy: got healthy result from https://10.10.0.171:2379 cluster is healthy