全手动搭建Kubernetes集群——集群规划及Etcd集群部署

一、集群规划

        k8s有两种集群架构模式:单Master集群和多Master集群;实际生产环境中需部署多Master集群,以确保高可用;目前由于资源限制和仅作为学习测试用,下面就搭建单Master集群架构。

        本人对单Master集群模式的机器规划:一台Master节点,三台Node工作节点,etcd数据库分别安装在Master、Node1和Node2节点上。

        各角色的分配以及需要安装的组件如下图:

watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBARG9ydGU3,size_20,color_FFFFFF,t_70,g_se,x_16

 

二、初始化操作

1.首先要关闭swap分区,这是从性能方面考虑的,因为当内存不足时,系统会自动调用swap分区,将部分内存数据存放到磁盘中,性能会下降。

watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBARG9ydGU3,size_20,color_FFFFFF,t_70,g_se,x_16

2.关闭防火墙:

[root@k8s-master-1 ~]# systemctl stop firewalld
[root@k8s-master-1 ~]# systemctl disable firewalld

3.关闭selinux:

[root@k8s-master-1 ~]# sed -i 's/enforcing/disabled/' /etc/selinux/config 

4.集群内所有机器都要进行时间同步:

[root@k8s-master-1 ~]# yum install -y ntpdate
[root@k8s-master-1 ~]# ntpdate time.windows.com
14 Jan 10:09:17 ntpdate[2597]: adjust time server 20.189.79.72 offset -0.008674 sec
[root@k8s-master-1 ~]# date
Fri Jan 14 10:09:30 EST 2022

三、Etcd集群搭建

         在Master、Node1和Node2上安装Etcd数据库。

1.下载cfssl工具来生成证书:

[root@k8s-master-1 ~]# curl -L https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -o /usr/local/bin/cfssl
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:03 --:--:--     0
100   654  100   654    0     0    157      0  0:00:04  0:00:04 --:--:--   157
100  9.8M  100  9.8M    0     0   133k      0  0:01:15  0:01:15 --:--:-- 89828

[root@k8s-master-1 ~]# curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /usr/local/bin/cfssljson
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   658  100   658    0     0    452      0  0:00:01  0:00:01 --:--:--   452
100 2224k  100 2224k    0     0   162k      0  0:00:13  0:00:13 --:--:--  154k

[root@k8s-master-1 ~]# curl -L https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 -o /usr/local/bin/cfssl-certinfo
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   663  100   663    0     0    566      0  0:00:01  0:00:01 --:--:--   566
100 6440k  100 6440k    0     0   313k      0  0:00:20  0:00:20 --:--:--  195k

        给文件执行权限:

[root@k8s-master-1 ~]# chmod +x /usr/local/bin/cfssl*

2.自签Etcd SSL证书:

        (1)需要创建以下目录:

  • /k8s/etcd/ssl(存放etcd的自签证书)
  • /k8s/etcd/cfg(存放etcd的配置文件)
  • /k8s/etcd/bin(存放etcd的执行程序)
[root@k8s-master-1 ~]# mkdir -p /k8s/etcd/{ssl,cfg,bin}

        (2)创建CA配置文件:ca-config.json

[root@k8s-master-1 ~]# cd /k8s/etcd/ssl
[root@k8s-master-1 ssl]# vim ca-config.json
[root@k8s-master-1 ssl]# cat ca-config.json 
{
  "signing": {    #表示该证书可用于签名其它证书,生成的ca.pem证书中的CA=TRUE
    "default": {    #可以定义多个profiles,分别指定不同的过期时间、使用场景等参数;后续在签名证书时使用某个profile
      "expiry": "87600h"    #证书过期时间
    },
    "profiles": {
      "etcd": {
        "usages": [
            "signing",
            "key encipherment",
            "server auth",    #表示client可以用该 CA 对server提供的证书进行验证
            "client auth"    #表示server可以用该CA对client提供的证书进行验证;
        ],
        "expiry": "87600h"
      }
    }
  }
}

        (3)创建CA证书签名请求文件:ca-scr.json

[root@k8s-master-1 ssl]# vim ca-csr.json 
[root@k8s-master-1 ssl]# cat ca-csr.json 
{
  "CN": "etcd",    #Common Name,kube-apiserver从证书中提取该字段作为请求的用户名 (User Name);浏览器使用该字段验证网站是否合法
  "key": {    #加密算法
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",    #国家
      "ST": "Shanghai",    #地区
      "L": "Shanghai",    #城市
      "O": "etcd",    #组织,kube-apiserver从证书中提取该字段作为请求用户所属的组(Group)
      "OU": "System"    #组织单位
    }
  ],
    "ca": {
       "expiry": "87600h"
    }
}

        (4)生成CA证书和私钥

[root@k8s-master-1 ssl]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca
2022/01/15 04:05:40 [INFO] generating a new CA key and certificate from CSR
2022/01/15 04:05:40 [INFO] generate received request
2022/01/15 04:05:40 [INFO] received CSR
2022/01/15 04:05:40 [INFO] generating key: rsa-2048
2022/01/15 04:05:40 [INFO] encoded CSR
2022/01/15 04:05:40 [INFO] signed certificate with serial number 429705018602797242754573367192655674978633104892

[root@k8s-master-1 ssl]# ll
total 20
-rw-r--r-- 1 root root  286 Jan 15 04:00 ca-config.json
-rw-r--r-- 1 root root  997 Jan 15 04:05 ca.csr
-rw-r--r-- 1 root root  250 Jan 15 04:01 ca-csr.json
-rw------- 1 root root 1679 Jan 15 04:05 ca-key.pem(CA私钥)
-rw-r--r-- 1 root root 1350 Jan 15 04:05 ca.pem(CA数字证书)

        (5)创建证书签名请求文件:etcd-csr.json

root@k8s-master-1 ssl]# cat > etcd-csr.json <<EOF
> {
>     "CN": "etcd",
>     "hosts": [    #需要指定授权使用该证书的主机IP或域名列表,这里配置所有Etcd的IP地址
>       "192.168.61.161",
>       "192.168.61.162",
>       "192.168.61.163"
>     ],
>     "key": {    #加密算法长度
>         "algo": "rsa",
>         "size": 2048
>     },
>     "names": [
>         {
>             "C": "CN",
>             "ST": "BeiJing",
>             "L": "BeiJing",
>             "O": "etcd",
>             "OU": "System"
>         }
>     ]
> }
> EOF

        为Etcd生成证书和私钥:

[root@k8s-master-1 ssl]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=etcd etcd-csr.json | cfssljson -bare etcd
2022/01/15 04:12:53 [INFO] generate received request
2022/01/15 04:12:53 [INFO] received CSR
2022/01/15 04:12:53 [INFO] generating key: rsa-2048
2022/01/15 04:12:53 [INFO] encoded CSR
2022/01/15 04:12:53 [INFO] signed certificate with serial number 169134289353272844556402247131396922483630866064
2022/01/15 04:12:53 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
  • etcd-key.pem:etcd的私钥
  • etcd.pem:etcd的数字证书

44ac0d9ab9e04c5085d0fa4dd74f12c7.png

         在后面部署Etcd集群时会用到下面几个关键的证书文件:

[root@k8s-master-1 ssl]# ls *pem
ca-key.pem  ca.pem  etcd-key.pem  etcd.pem

3.Etcd数据库集群部署

        Etcd数据库集群采用主从架构模式部署(一主多从),部署奇数台节点,集群会通过选举产生Leader。使用Raft一致性算法保证每个节点的一致性。

watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBARG9ydGU3,size_20,color_FFFFFF,t_70,g_se,x_16

         (1)下载etcd,从GitHub上下载合适版本的etcd,并且解压:

[root@k8s-master-1 ~]# cd /k8s/etcd/
[root@k8s-master-1 etcd]# wget https://github.com/etcd-io/etcd/releases/download/v3.2.28/etcd-v3.2.28-linux-amd64.tar.gz

[root@k8s-master-1 etcd]# tar zxf etcd-v3.2.28-linux-amd64.tar.gz

                将etcd复制到/usr/local/bin目录下:

[root@k8s-master-1 etcd]# cp etcd-v3.2.28-linux-amd64/{etcd,etcdctl} /k8s/etcd/bin

        (2)创建Etcd配置文件:etcd.conf

[root@k8s-master-1 ~]# cd /k8s/etcd/cfg/
[root@k8s-master-1 cfg]# vim etcd.conf 
[root@k8s-master-1 cfg]# cat etcd.conf 
# [member]
ETCD_NAME=etcd-1
ETCD_DATA_DIR=/k8s/data/default.etcd
ETCD_LISTEN_PEER_URLS=https://192.168.61.161:2380
ETCD_LISTEN_CLIENT_URLS=https://192.168.61.161:2379

# [cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS=https://192.168.61.161:2380
ETCD_ADVERTISE_CLIENT_URLS=https://192.168.61.161:2379
ETCD_INITIAL_CLUSTER=etcd-1=https://192.168.61.161:2380,etcd-2=https://192.168.61.162:2380,etcd-3=https://192.168.61.163:2380
ETCD_INITIAL_CLUSTER_TOKEN=etcd-cluster
ETCD_INITIAL_CLUSTER_STATE=new

# [security]
ETCD_CERT_FILE=/k8s/etcd/ssl/etcd.pem
ETCD_KEY_FILE=/k8s/etcd/ssl/etcd-key.pem
ETCD_TRUSTED_CA_FILE=/k8s/etcd/ssl/ca.pem
ETCD_PEER_CERT_FILE=/k8s/etcd/ssl/etcd.pem
ETCD_PEER_KEY_FILE=/k8s/etcd/ssl/etcd-key.pem
ETCD_PEER_TRUSTED_CA_FILE=/k8s/etcd/ssl/ca.pem

        (3)创建Etcd服务:etcd.service

[root@k8s-master-1 etcd]# vim etcd.service
[root@k8s-master-1 etcd]# cat etcd.service 
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=/k8s/etcd/cfg/etcd.conf    #指定etcd.conf作为环境配置文件
WorkingDirectory=${ETCD_DATA_DIR}

ExecStart=/k8s/etcd/bin/etcd \
  --name=${ETCD_NAME} \
  --data-dir=${ETCD_DATA_DIR} \
  --listen-peer-urls=${ETCD_LISTEN_PEER_URLS} \
  --listen-client-urls=${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 \
  --initial-advertise-peer-urls=${ETCD_INITIAL_ADVERTISE_PEER_URLS} \
  --advertise-client-urls=${ETCD_ADVERTISE_CLIENT_URLS} \
  --initial-cluster=${ETCD_INITIAL_CLUSTER} \
  --initial-cluster-token=${ETCD_INITIAL_CLUSTER_TOKEN} \
  --initial-cluster-state=${ETCD_INITIAL_CLUSTER_STATE} \
  --cert-file=${ETCD_CERT_FILE} \
  --key-file=${ETCD_KEY_FILE} \
  --trusted-ca-file=${ETCD_TRUSTED_CA_FILE} \
  --peer-cert-file=${ETCD_PEER_CERT_FILE} \
  --peer-key-file=${ETCD_PEER_KEY_FILE} \
  --peer-trusted-ca-file=${ETCD_PEER_TRUSTED_CA_FILE}

Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

        (4)将Etcd目录拷贝到另外两个节点:

[root@k8s-master-1 etcd]# scp -r /k8s root@k8s-node-1:/k8s

[root@k8s-master-1 etcd]# scp -r /k8s root@k8s-node-2:/k8s

        (5)修改两个节点的etcd.conf配置文件:

[root@k8s-node-1 cfg]# vim etcd.conf 
[root@k8s-node-1 cfg]# cat etcd.conf 
# [member]
ETCD_NAME=etcd-2
ETCD_DATA_DIR=/k8s/data/default.etcd
ETCD_LISTEN_PEER_URLS=https://192.168.61.162:2380
ETCD_LISTEN_CLIENT_URLS=https://192.168.61.162:2379

# [cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS=https://192.168.61.162:2380
ETCD_ADVERTISE_CLIENT_URLS=https://192.168.61.162:2379
ETCD_INITIAL_CLUSTER=etcd-1=https://192.168.61.161:2380,etcd-2=https://192.168.61.162:2380,etcd-3=https://192.168.61.163:2380
ETCD_INITIAL_CLUSTER_TOKEN=etcd-cluster
ETCD_INITIAL_CLUSTER_STATE=new

# [security]
ETCD_CERT_FILE=/k8s/etcd/ssl/etcd.pem
ETCD_KEY_FILE=/k8s/etcd/ssl/etcd-key.pem
ETCD_TRUSTED_CA_FILE=/k8s/etcd/ssl/ca.pem
ETCD_PEER_CERT_FILE=/k8s/etcd/ssl/etcd.pem
ETCD_PEER_KEY_FILE=/k8s/etcd/ssl/etcd-key.pem
ETCD_PEER_TRUSTED_CA_FILE=/k8s/etcd/ssl/ca.pem



[root@k8s-node-2 cfg]# vim etcd.conf 
[root@k8s-node-2 cfg]# cat etcd.conf 
# [member]
ETCD_NAME=etcd-3
ETCD_DATA_DIR=/k8s/data/default.etcd
ETCD_LISTEN_PEER_URLS=https://192.168.61.163:2380
ETCD_LISTEN_CLIENT_URLS=https://192.168.61.163:2379

# [cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS=https://192.168.61.163:2380
ETCD_ADVERTISE_CLIENT_URLS=https://192.168.61.163:2379
ETCD_INITIAL_CLUSTER=etcd-1=https://192.168.61.161:2380,etcd-2=https://192.168.61.162:2380,etcd-3=https://192.168.61.163:2380
ETCD_INITIAL_CLUSTER_TOKEN=etcd-cluster
ETCD_INITIAL_CLUSTER_STATE=new

# [security]
ETCD_CERT_FILE=/k8s/etcd/ssl/etcd.pem
ETCD_KEY_FILE=/k8s/etcd/ssl/etcd-key.pem
ETCD_TRUSTED_CA_FILE=/k8s/etcd/ssl/ca.pem
ETCD_PEER_CERT_FILE=/k8s/etcd/ssl/etcd.pem
ETCD_PEER_KEY_FILE=/k8s/etcd/ssl/etcd-key.pem
ETCD_PEER_TRUSTED_CA_FILE=/k8s/etcd/ssl/ca.pem

        (6)启动Etcd服务:将etcd.service拷贝到/usr/lib/systemd/system/目录下,启动Etcd服务

[root@k8s-master-1 etcd]# cp /k8s/etcd/etcd.service /usr/lib/systemd/system/
[root@k8s-master-1 etcd]# systemctl daemon-reload
[root@k8s-master-1 etcd]# systemctl start etcd
[root@k8s-master-1 etcd]# systemctl enable etcd


[root@k8s-node-1 etcd]# cp /k8s/etcd/etcd.service /usr/lib/systemd/system/
[root@k8s-node-1 etcd]# systemctl daemon-reload
[root@k8s-node-1 etcd]# systemctl start etcd
[root@k8s-node-1 etcd]# systemctl enable etcd

[root@k8s-node-2 etcd]# cp /k8s/etcd/etcd.service /usr/lib/systemd/system/
[root@k8s-node-2 etcd]# systemctl daemon-reload
[root@k8s-node-2 etcd]# systemctl start etcd
[root@k8s-node-2 etcd]# systemctl enable etcd

        查看Etcd的日志命令:tail -f /var/log/messages

        查看Etcd集群状态命令:/k8s/etcd/bin/etcdctl --ca-file=/k8s/etcd/ssl/ca.pem --cert-file=/k8s/etcd/ssl/etcd.pem --key-file=/k8s/etcd/ssl/etcd-key.pem --endpoints=https://192.168.61.161:2379,https://192.168.61.162:2379,https://192.168.61.163:2379 cluster-health

[root@k8s-master-1 etcd]# /k8s/etcd/bin/etcdctl --ca-file=/k8s/etcd/ssl/ca.pem --cert-file=/k8s/etcd/ssl/etcd.pem --key-file=/k8s/etcd/ssl/etcd-key.pem --endpoints=https://192.168.61.161:2379,https://192.168.61.162:2379,https://192.168.61.163:2379 cluster-health
member 4166e968fa162f83 is healthy: got healthy result from https://192.168.61.161:2379
member cdade95107acd749 is healthy: got healthy result from https://192.168.61.163:2379
member cecb7c331cf85085 is healthy: got healthy result from https://192.168.61.162:2379
cluster is healthy

        到此,完成k8s的Etcd数据库集群部署!下一篇文章将进行Master管理节点和Node工作节点的部署。

 

 

 

 

  • 3
    点赞
  • 7
    收藏
    觉得还不错? 一键收藏
  • 2
    评论
评论 2
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值