Kubernetes（二）集群部署---etcd

Kubernetes（二）集群部署

一、前景：

1. 官方提供的三种部署方式

2. Kubernetes平台环境规划

3. 自签SSL证书

4. Etcd数据库集群部署

5. Node安装Docker

6. Flannel容器集群网络部署

7. 部署Master组件

8. 部署Node组件

9. 部署一个测试示例

10. 部署Web UI (Dashboard)

11. 部署集群内部DNS解析服务(CoreDNS)

二、K8S全部配完所需的主机

2台master 2-2

2台node 2-4/2-3

2台lb 1-2

1台harbor 1-2

三、自签 SSL 证书

组件	使用证书
etcd	ca.pem、server.pem、server-key.pem
fianne	ca.pem、server.pem、server-key.pem
kube-apiserver	ca.pem、server.pem、server-key.pem
kubelet	ca.pem、ca-key.pem
kube-proxy	ca.pem、kube-proxy.pem、kube-proxy-key.pem
kubectl	ca.pem、admin.pem、admin-key.pem

四、Kubernetes二进制部署

环境部署

二进制包：

从官方下载二进制包，下载地址：https://github.com/kubernetes/kubernetes/releases

主机	IP	配置	组件
master1	192.168.171.7	2/2	kube-apiserver、kube-controller-manager、kube-scheduler、etcd
node1	192.168.171.4	2/3	kubelet、kube-proxy、docker、flannel、etcd
node2	192.168.171.17	2/3	kubelet、kube-proxy、docker、flannel、etcd

先给各个主机命名

hostnamectl set-hostname ...
su

（1）master1

[root@localhost ~]# hostnamectl set-hostname Master1
[root@localhost ~]# su
[root@master1 ~]# mkdir k8s
[root@master1 ~]# cd k8s/
[root@master1 k8s]# ls     
etcd-cert.sh  etcd.sh     ←//上传两个对应的软件包
[root@master1 k8s]# mkdir etcd-cert
[root@master1 k8s]# mv etcd-cert.sh etcd-cert    #移动到文件夹下
[root@master1 k8s]# ls
etcd-cert  etcd.sh

下载证书制作工具

#创建脚本编辑添加
vim cfssl.sh

curl -L https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -o /usr/local/bin/cfssl
curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /usr/local/bin/cfssljson
curl -L https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 -o /usr/local/bin/cfssl-certinfo
chmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson /usr/local/bin/cfssl-certinfo

#执行上方脚本下载cfssl官方包
[root@master1 k8s]# bash cfssl.sh
[root@master1 k8s]# ls /usr/local/bin/   #查看包
cfssl  cfssl-certinfo  cfssljson

开始制作证书

• cfssl 生成证书工具
• cfssljson通过传入json文件生成证书
• cfssl-certinfo查看证书信息

定义ca证书

cd  /root/k8s/etcd-cert

cat > ca-config.json <<EOF
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "www": {
         "expiry": "87600h",
         "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"     
        ]  
      } 
    }         
  }
}
EOF

实现证书签名

cat > ca-csr.json <<EOF 
{   
    "CN": "etcd CA",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Beijing",
            "ST": "Beijing"
        }
    ]
}
EOF

生产证书，生成ca-key.pem ca.pem

[root@master1 etcd-cert]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca -

2021/04/12 12:29:43 [INFO] generating a new CA key and certificate from CSR
2021/04/12 12:29:43 [INFO] generate received request
2021/04/12 12:29:43 [INFO] received CSR
2021/04/12 12:29:43 [INFO] generating key: rsa-2048
2021/04/12 12:29:43 [INFO] encoded CSR
2021/04/12 12:29:43 [INFO] signed certificate with serial number 276243159681123507007027000628133507907973178665

指定etcd三个节点之间的通信验证

cat > server-csr.json <<EOF
{
    "CN": "etcd",
    "hosts": [
    "192.168.171.7",
    "192.168.171.4",
    "192.168.171.17"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "BeiJing",
            "ST": "BeiJing"
        }
    ]
}
EOF

生成ETCD证书 server-key.pem server.pem

命令：
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server

上传以下压缩包到/root/k8s目录当中

etcd-v3.3.10-linux-amd64.tar.gz

flannel-v0.10.0-linux-amd64.tar.gz

kubernetes-server-linux-amd64.tar.gz

解压文件
tar zxvf etcd-v3.3.10-linux-amd64.tar.gz

[root@master1 k8s]# ls etcd-v3.3.10-linux-amd64
Documentation  etcdctl            README.md
etcd           README-etcdctl.md  READMEv2-etcdctl.md
[root@master1 k8s]# mkdir /opt/etcd/{cfg,bin,ssl} -p	//为etcd创建工作目录：配置文件、命令文件、证书
[root@master1 k8s]# mv etcd-v3.3.10-linux-amd64/etcd etcd-v3.3.10-linux-amd64/etcdctl /opt/etcd/bin/		//将启动和管理移动至命令文件目录下
[root@master1 k8s]# ls /opt/etcd/bin/
etcd  etcdctl

证书拷贝

[root@master1 k8s]#  cp etcd-cert/*.pem /opt/etcd/ssl/
[root@master1 k8s]# ls /opt/etcd/ssl/
ca-key.pem  ca.pem  server-key.pem  server.pem

进入卡住状态等待其他节点加入

bash etcd.sh etcd01 192.168.171.7 etcd02=https://192.168.171.4:2380,etcd03=https://192.168.171.17:2380

会超时，是正常现象，我们接着往下配

拷贝证书去其他节点

scp -r /opt/etcd/ root@192.168.171.4:/opt/
scp -r /opt/etcd/ root@192.168.171.17:/opt/

启动脚本拷贝其他节点

scp /usr/lib/systemd/system/etcd.service root@192.168.171.4:/usr/lib/systemd/system/

scp /usr/lib/systemd/system/etcd.service root@192.168.171.17:/usr/lib/systemd/system/

（2）node1节点和node2节点都需要修改

node1

cd /opt/etcd/cfg/
vim etcd 

node2

启动

在master1启动

cd /root/k8s/

bash etcd.sh etcd01 192.168.171.7 etcd02=https://192.168.171.4:2380,etcd03=https://192.168.171.17:2380

在node1和node2也分别启动

systemctl start etcd.service 

在master1上检查集群状态

cd /root/k8s/etcd-cert/

/opt/etcd/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://192.168.171.7:2379,https://192.168.171.4:2379,https://192.168.171.17:2379" cluster-health