k8s二进制单节点部署

最新推荐文章于 2022-03-26 08:50:53 发布

weixin_49226813

最新推荐文章于 2022-03-26 08:50:53 发布

阅读量290

点赞数

分类专栏： k8s 文章标签： k8s

本文链接：https://blog.csdn.net/weixin_49226813/article/details/112846486

版权

k8s 专栏收录该内容

3 篇文章 0 订阅

订阅专栏

部署规划

在这里插入图片描述

名称	主机名称	IP地址
master节点	master	20.0.0.10/24
node节点	node01	20.0.0.20/24
node节点	node02	20.0.0.30/24

一、etcd数据库部署

master节点部署

1、创建目录

[root@master ~]# mkdir k8s
[root@master ~]# cd k8s/
[root@master k8s]# mkdir etcd-cert
[root@master k8s]# cd etcd-cert/

2、制作ca证书

上传制作证书的工具
[root@master etcd-cert]# ls
cfssl  cfssl-certinfo  cfssljson
[root@master etcd-cert]# chmod +x *
[root@master etcd-cert]# ll
总用量 18808
-rwxr-xr-x 1 root root 10376657 1月  16 2020 cfssl
-rwxr-xr-x 1 root root  6595195 1月  16 2020 cfssl-certinfo
-rwxr-xr-x 1 root root  2277873 1月  16 2020 cfssljson
[root@master etcd-cert]# mv cf* /usr/local/bin/

开始制作证书
1、定义ca证书
[root@master etcd-cert]# vim ca-config.json
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "www": {
         "expiry": "87600h",
         "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ]
      }
    }
  }
}

2、设置ca证书签名
[root@master etcd-cert]# vim ca-csr.json
{
    "CN": "etcd CA",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Beijing",
            "ST": "Beijing"
        }
    ]
}

3、生成ca证书
[root@master etcd-cert]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
2021/01/19 16:58:40 [INFO] generating a new CA key and certificate from CSR
2021/01/19 16:58:40 [INFO] generate received request
2021/01/19 16:58:40 [INFO] received CSR
2021/01/19 16:58:40 [INFO] generating key: rsa-2048
2021/01/19 16:58:40 [INFO] encoded CSR
2021/01/19 16:58:40 [INFO] signed certificate with serial number 80068717008280235134541270449892611068755928092
[root@master etcd-cert]# ll
总用量 20
-rw-r--r-- 1 root root  304 1月  19 16:55 ca-config.json
-rw-r--r-- 1 root root  956 1月  19 16:58 ca.csr
-rw-r--r-- 1 root root  212 1月  19 16:57 ca-csr.json
-rw------- 1 root root 1679 1月  19 16:58 ca-key.pem
-rw-r--r-- 1 root root 1265 1月  19 16:58 ca.pem		#生成的ca证书

3、设置etcd通讯验证

[root@master etcd-cert]# vim server-csr.json
{
    "CN": "etcd",
    "hosts": [
    "20.0.0.10",
    "20.0.0.20",
    "20.0.0.30"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "BeiJing",
            "ST": "BeiJing"
        }
    ]
}

生成etcd证书
[root@master etcd-cert]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server
2021/01/19 17:05:14 [INFO] generate received request
2021/01/19 17:05:14 [INFO] received CSR
2021/01/19 17:05:14 [INFO] generating key: rsa-2048
2021/01/19 17:05:15 [INFO] encoded CSR
2021/01/19 17:05:15 [INFO] signed certificate with serial number 274602113254117039354472428397076843843685311146
2021/01/19 17:05:15 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
[root@master etcd-cert]# ll
总用量 36
-rw-r--r-- 1 root root  304 1月  19 16:55 ca-config.json
-rw-r--r-- 1 root root  956 1月  19 16:58 ca.csr
-rw-r--r-- 1 root root  212 1月  19 16:57 ca-csr.json
-rw------- 1 root root 1679 1月  19 16:58 ca-key.pem
-rw-r--r-- 1 root root 1265 1月  19 16:58 ca.pem
-rw-r--r-- 1 root root 1013 1月  19 17:05 server.csr
-rw-r--r-- 1 root root  278 1月  19 17:05 server-csr.json
-rw------- 1 root root 1679 1月  19 17:05 server-key.pem
-rw-r--r-- 1 root root 1338 1月  19 17:05 server.pem
[root@master etcd-cert]#

4、创建etcd的工作目录

[root@master etcd-cert]# mkdir /opt/etcd/{cfg,bin,ssl} -p
cfg：放tecd配置文件
bin：放etcd命令工具
ssl：放etcd证书

[root@master etcd-cert]# cp etcd-v3.3.10-linux-amd64/etcd etcd-v3.3.10-linux-amd64/etcdctl /opt/etcd/bin/
[root@master etcd-cert]# cp *.pem /opt/etcd/ssl/

5、设置etcd配置文件和启动脚本

[root@master etcd-cert]# vim etcd.sh 
#!/bin/bash# example: ./etcd.sh etcd01 20.0.0.10 etcd02=https://20.0.0.20:2380,etcd03=https://20.0.0.30:2380

ETCD_NAME=$1
ETCD_IP=$2
ETCD_CLUSTER=$3

WORK_DIR=/opt/etcd

cat <<EOF >$WORK_DIR/cfg/etcd
#[Member]
ETCD_NAME="${ETCD_NAME}"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://${ETCD_IP}:2380"
ETCD_LISTEN_CLIENT_URLS="https://${ETCD_IP}:2379"

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://${ETCD_IP}:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://${ETCD_IP}:2379"
ETCD_INITIAL_CLUSTER="etcd01=https://${ETCD_IP}:2380,${ETCD_CLUSTER}"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF

cat <<EOF >/usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=${WORK_DIR}/cfg/etcd
ExecStart=${WORK_DIR}/bin/etcd \
--name=\${ETCD_NAME} \
--data-dir=\${ETCD_DATA_DIR} \
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=${WORK_DIR}/cfg/etcd
ExecStart=${WORK_DIR}/bin/etcd \
--name=\${ETCD_NAME} \
--data-dir=\${ETCD_DATA_DIR} \
--listen-peer-urls=\${ETCD_LISTEN_PEER_URLS} \
--listen-client-urls=\${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 \
--advertise-client-urls=\${ETCD_ADVERTISE_CLIENT_URLS} \
--initial-advertise-peer-urls=\${ETCD_INITIAL_ADVERTISE_PEER_URLS} \
--initial-cluster=\${ETCD_INITIAL_CLUSTER} \
--initial-cluster-token=\${ETCD_INITIAL_CLUSTER_TOKEN} \
--initial-cluster-state=new \
--cert-file=${WORK_DIR}/ssl/server.pem \
--key-file=${WORK_DIR}/ssl/server-key.pem \
--peer-cert-file=${WORK_DIR}/ssl/server.pem \
--peer-key-file=${WORK_DIR}/ssl/server-key.pem \
--trusted-ca-file=${WORK_DIR}/ssl/ca.pem \
--peer-trusted-ca-file=${WORK_DIR}/ssl/ca.pem
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable etcd
systemctl restart etcd

执行脚本
[root@master etcd-cert]# bash etcd.sh etcd01 20.0.0.10 etcd02=https://20.0.0.20:2380,etcd03=https://20.0.0.30:2380
处于等待状态，应为node节点还没加入进来，此时应该生成etcd的配置文件和启动文件
#安装tree来查看etcd工作目录
[root@master etcd-cert]# yum -y install tree
[root@master etcd-cert]# tree /opt/etcd/
/opt/etcd/
├── bin
│   ├── etcd
│   └── etcdctl
├── cfg
│   └── etcd
└── ssl
    ├── ca-key.pem
    ├── ca.pem
    ├── server-key.pem
    └── server.pem

node节点部署

node01节点部署

#在master节点将etcd的工作目录文件拷贝到node01的/opt目录下面
[root@master etcd]# scp -r /opt/etcd/ root@20.0.0.20:/opt/
The authenticity of host '20.0.0.20 (20.0.0.20)' can't be established.
ECDSA key fingerprint is SHA256:yELtpt+yAiWNtPQb5bPu3PyWman5X5xL5zwU607sqHE.
ECDSA key fingerprint is MD5:b1:3f:1c:fa:eb:42:fc:bf:02:40:80:ea:8e:01:0a:7a.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '20.0.0.20' (ECDSA) to the list of known hosts.
root@20.0.0.20's password: 
etcd                                                             100%  481   152.4KB/s   00:00    
etcd                                                             100%   18MB  68.1MB/s   00:00    
etcdctl                                                          100%   15MB  74.6MB/s   00:00    
ca-key.pem                                                       100% 1679     1.4MB/s   00:00    
ca.pem                                                           100% 1265     1.1MB/s   00:00    
server-key.pem                                                   100% 1679     1.7MB/s   00:00    
server.pem                                                       100% 1338     1.1MB/s   00:00    

#在master节点将etcd拷贝启动配置文件给node01
[root@master etcd]# scp /usr/lib/systemd/system/etcd.service root@20.0.0.20:/usr/lib/systemd/system/etcd.service 
root@20.0.0.20's password: 
etcd.service 

#修改/opt/etcd/cfg/etcd文件里的IP地址和节点名称
[root@node01 ~]# vim /opt/etcd/cfg/etcd
#[Member]
ETCD_NAME="etcd02"	#修改etcd02
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://20.0.0.20:2380"	#修改为自己主机IP地址
ETCD_LISTEN_CLIENT_URLS="https://20.0.0.20:2379"	#修改为自己主机IP地址

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://20.0.0.20:2380"	#修改为自己主机IP地址
ETCD_ADVERTISE_CLIENT_URLS="https://20.0.0.20:2379"	#修改为自己主机IP地址
ETCD_INITIAL_CLUSTER="etcd01=https://20.0.0.10:2380,etcd02=https://20.0.0.20:2380,etcd03=https://20.0.0.30:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"


node02
#在master节点将etcd的工作目录文件拷贝到node02的/opt目录下面
[root@master etcd]# scp -r /opt/etcd/ root@20.0.0.30:/opt/
The authenticity of host '20.0.0.30 (20.0.0.30)' can't be established.
ECDSA key fingerprint is SHA256:S2ANKK3sAHs5II74zKqBXVTVfAQUFrImRm7pq/hOrpg.
ECDSA key fingerprint is MD5:bc:99:68:b2:2a:25:31:37:bc:2b:01:49:35:98:de:f0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '20.0.0.30' (ECDSA) to the list of known hosts.
root@20.0.0.30's password: 
etcd                                                             100%  481   194.1KB/s   00:00    
etcd                                                             100%   18MB  92.3MB/s   00:00    
etcdctl                                                          100%   15MB  93.0MB/s   00:00    
ca-key.pem                                                       100% 1679   745.9KB/s   00:00    
ca.pem                                                           100% 1265   416.2KB/s   00:00    
server-key.pem                                                   100% 1679   852.3KB/s   00:00    
server.pem

#在master节点将etcd拷贝启动配置文件给node02
[root@master etcd]# scp /usr/lib/systemd/system/etcd.service root@20.0.0.30:/usr/lib/systemd/system/etcd.service 
root@20.0.0.30's password: 
etcd.service

#修改/opt/etcd/cfg/etcd文件里的IP地址和节点名称
[root@node01 ~]# vim /opt/etcd/cfg/etcd
#[Member]
ETCD_NAME="etcd03"	#修改etcd03
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://20.0.0.30:2380"	#修改为自己主机IP地址
ETCD_LISTEN_CLIENT_URLS="https://20.0.0.30:2379"	#修改为自己主机IP地址

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://20.0.0.30:2380"	#修改为自己主机IP地址
ETCD_ADVERTISE_CLIENT_URLS="https://20.0.0.30:2379"	#修改为自己主机IP地址
ETCD_INITIAL_CLUSTER="etcd01=https://20.0.0.10:2380,etcd02=https://20.0.0.20:2380,etcd03=https://20.0.0.30:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

node节点加入群集

#先启动master节点的etcd在启动node节点etcd
[root@master etcd]# systemctl start etcd
[root@node01 etcd]# systemctl start etcd
[root@node01 etcd]# systemctl start etcd

查看群集是否成功加入
[root@master etcd]# cd /opt/etcd/ssl/
[root@master ssl]# /opt/etcd/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://20.0.0.10:2379,https://20.0.0.20:2379,https://20.0.0.30:2379" cluster-health
member 60bc7e36f63b965 is healthy: got healthy result from https://20.0.0.30:2379
member 4a9f1750486efa02 is healthy: got healthy result from https://20.0.0.10:2379
member e3197fd6a5933614 is healthy: got healthy result from https://20.0.0.20:2379
cluster is healthy

二、docker容器引擎部署

所有从节点部署docker

1、下载阿里源

yum-config-manager     --add-repo     http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

2、安装docker依赖环境

yum install -y yum-utils device-mapper-persistent-data lvm2

3、安装docker-ce

yum -y install docker-ce

4、启动docker

systemctl start docker.service 
systemctl enable docker.service	#设置开启自启动

三、flannel网络组件部署

master分配子网写入etcd

1、写入分配的子网段到tecd中，供flannel使用

cd /opt/etcd/ssl/

/opt/etcd/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://20.0.0.10:2379,https://20.0.0.20:2379,https://20.0.0.30:2379" set /coreos.com/network/config '{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}'

{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}		#出现这个表明成功

2、查看信息是否写入etcd

/opt/etcd/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://20.0.0.10:2379,https://20.0.0.20:2379,https://20.0.0.30:2379" get /coreos.com/network/config

{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}		#出现这个表面写入成功

node节点安装flannel

所有node节点安装flannel

1、上传flannel安装包

ll
-rw-r--r--  1 root root 9706487 1月  13 2020 flannel-v0.10.0-linux-amd64.tar.gz

tar zxvf flannel-v0.10.0-linux-amd64.tar.gz 

ll
-rwxr-xr-x  1 1001 1001 36327752 1月  24 2018 flanneld
-rwxr-xr-x  1 1001 1001     2139 3月  18 2017 mk-docker-opts.sh
-rw-rw-r--  1 1001 1001     4298 12月 24 2017 README.md

2、创建flannel工作目录

mkdir /opt/kubernetes/{cfg,bin,ssl} -p

3、flannel工具移动到创建的工作目录中

mv flanneld mk-docker-opts.sh /opt/kubernetes/bin/

4、创建flannel配置、启动脚本

vim flannel.sh
#!/bin/bash

ETCD_ENDPOINTS=${1:-"http://127.0.0.1:2379"}

cat <<EOF >/opt/kubernetes/cfg/flanneld

FLANNEL_OPTIONS="--etcd-endpoints=${ETCD_ENDPOINTS} \
-etcd-cafile=/opt/etcd/ssl/ca.pem \
-etcd-certfile=/opt/etcd/ssl/server.pem \
-etcd-keyfile=/opt/etcd/ssl/server-key.pem"

EOF

cat <<EOF >/usr/lib/systemd/system/flanneld.service
[Unit]
Description=Flanneld overlay address etcd agent
After=network-online.target network.target
Before=docker.service

[Service]
Type=notify
EnvironmentFile=/opt/kubernetes/cfg/flanneld
ExecStart=/opt/kubernetes/bin/flanneld --ip-masq \$FLANNEL_OPTIONS
ExecStartPost=/opt/kubernetes/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/subnet.env
Restart=on-failure

[Install]
WantedBy=multi-user.target

EOF

systemctl daemon-reload
systemctl enable flanneld
systemctl restart flanneld

5、开启flannel

bash flannel.sh https://20.0.0.10:2379,https://20.0.0.20:2379,https://20.0.0.30:2379

此时node节点都会出现一个flannel网卡，而且每个节点的flannel网卡的网段都不相同

6、配置docker连接flannel

vim /usr/lib/systemd/system/docker.service
在[Service]模块下
EnvironmentFile=/run/flannel/subnet.env	#增加
ExecStart=/usr/bin/dockerd $DOCKER_NETWORK_OPTIONS -H fd:// --containerd=/run/containerd/containerd.sock	在-H前面增加 $DOCKER_NETWORK_OPTIONS

#查看$DOCKER_NETWORK_OPTIONS是什么
cat /run/flannel/subnet.env 
DOCKER_OPT_BIP="--bip=172.17.100.1/24"
DOCKER_OPT_IPMASQ="--ip-masq=false"
DOCKER_OPT_MTU="--mtu=1450"
DOCKER_NETWORK_OPTIONS=" --bip=172.17.100.1/24 --ip-masq=false --mtu=1450"	#--bip指定docker启动时的子网段，每个node节点都不相同

7、重启docker

systemctl daemon-reload 	#重新加载系统参数
systemctl restart docker.service

8、测试不同节点之间docker网络是否可以正常通讯

1、node01创建一个容器
[root@node01 ~]# docker run -itd centos:7

2、node01创建一个容器
[root@node02 ~]# docker run -itd centos:7

3、查看node01节点容器IP地址
[root@node01 ~]# docker ps -a
CONTAINER ID   IMAGE      COMMAND       CREATED         STATUS         PORTS     NAMES
b74642711c2f   centos:7   "/bin/bash"   3 minutes ago   Up 3 minutes             elastic_leakey
[root@node01 ~]# docker inspect --format={{.NetworkSettings.IPAddress}} b74642711c2f
172.17.100.2

4、查看node02节点容器IP地址
[root@node2 ~]# docker ps -a
CONTAINER ID   IMAGE      COMMAND       CREATED              STATUS              PORTS     NAMES
43a46bd8868f   centos:7   "/bin/bash"   About a minute ago   Up About a minute             nice_driscoll
[root@node2 ~]# docker inspect --format={{.NetworkSettings.IPAddress}} 43a46bd8868f 
172.17.75.2

5、node01节点容器ping node02节点容器
[root@node01 ~]# docker exec -it b74642711c2f ping 172.17.75.2
PING 172.17.75.2 (172.17.75.2) 56(84) bytes of data.
64 bytes from 172.17.75.2: icmp_seq=1 ttl=62 time=0.618 ms
64 bytes from 172.17.75.2: icmp_seq=2 ttl=62 time=1.33 ms
64 bytes from 172.17.75.2: icmp_seq=3 ttl=62 time=0.758 ms

6、node02节点容器ping node01节点容器
[root@node2 ~]# docker exec -it 43a46bd8868f ping 172.17.100.2
PING 172.17.100.2 (172.17.100.2) 56(84) bytes of data.
64 bytes from 172.17.100.2: icmp_seq=1 ttl=62 time=0.469 ms
64 bytes from 172.17.100.2: icmp_seq=2 ttl=62 time=0.505 ms
64 bytes from 172.17.100.2: icmp_seq=3 ttl=62 time=1.31 ms

四、master节点部署

1、创建kubernetes工作目录

[root@master ~]# mkdir /opt/kubernetes/{cfg,bin,ssl} -p

2、制作api-server证书

[root@master ~]# mkdir k8s-cert
[root@master ~]# cd k8s-cert/
#######################制作ca证书############################
[root@master k8s-cert]#vim ca-config.json	#配置ca证书
{
 "signing": {
   "default": {
     "expiry": "87600h"
   },
   "profiles": {
     "kubernetes": {
        "expiry": "87600h",
        "usages": [
           "signing",
           "key encipherment",
           "server auth",
           "client auth"
       ]
     }
   }
 }
}

[root@master k8s-cert]#vim ca-csr.json		#ca证书签名
{
   "CN": "kubernetes",
   "key": {
       "algo": "rsa",
       "size": 2048
   },
   "names": [
       {
           "C": "CN",
           "L": "Beijing",
           "ST": "Beijing",
           "O": "k8s",
           "OU": "System"
       }
   ]
}

[root@master k8s-cert]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca -	生成ca证书

#######################api-server############################
[root@master k8s-cert]# vim server-csr.json
{
   "CN": "kubernetes",
   "hosts": [
     "10.0.0.1",
     "127.0.0.1",
     "20.0.0.10",      #master01 节点IP，就是本机IP地址
     "20.0.0.40", #master02 节点IP，为k8s多节点部署提前设置
     "20.0.0.50", #nginx-master
     "20.0.0.60", #nginx-back
     "20.0.0.100", #VIP地址
     "kubernetes",
     "kubernetes.default",
     "kubernetes.default.svc",
     "kubernetes.default.svc.cluster",
     "kubernetes.default.svc.cluster.local"
   ],
   "key": {
       "algo": "rsa",
       "size": 2048
   },
   "names": [
       {
           "C": "CN",
           "L": "BeiJing",
           "ST": "BeiJing",
           "O": "k8s",
           "OU": "System"
       }
   ]
}

[root@master k8s-cert]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server		#生成api-server证书

#######################管理员角色证书############################
[root@master k8s-cert]# vim admin-csr.json
{
 "CN": "admin",
 "hosts": [],
 "key": {
   "algo": "rsa",
   "size": 2048
 },
 "names": [
   {
     "C": "CN",
     "L": "BeiJing",
     "ST": "BeiJing",
     "O": "system:masters",
     "OU": "System"
   }
 ]
}

[root@master k8s-cert]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin		#生成管理员证书

#######################kube-proxy证书############################
[root@master k8s-cert]# vim kube-proxy.json
{
 "CN": "system:kube-proxy",
 "hosts": [],
 "key": {
   "algo": "rsa",
   "size": 2048
 },
 "names": [
   {
     "C": "CN",
     "L": "BeiJing",
     "ST": "BeiJing",
     "O": "k8s",
     "OU": "System"
   }
 ]
}

[root@master k8s-cert]#  cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy		生成kube-proxy证书

[root@master k8s-cert]# ls *.pem
admin-key.pem  admin.pem  ca-key.pem  ca.pem  kube-proxy-key.pem  kube-proxy.pem  server-key.pem  server.pem

3、将证书移动到制定位置

[root@master k8s-cert]# cp *.pem /opt/kubernetes/ssl/

4、上传k8s master核心组件包

[root@master k8s-cert# cd /root/k8s/
[root@master k8s]# ll
总用量 412700
drwxr-xr-x 3 root root       260 1月  19 17:49 etcd-cert
-rw-r--r-- 1 root root 422603860 1月  12 2020 kubernetes-server-linux-amd64.tar.gz

[root@master k8s]# tar zxvf kubernetes-server-linux-amd64.tar.gz
[root@master k8s]# cd kubernetes/
[root@master kubernetes]# cd server/
[root@master server]# cd bin/
[root@master bin]# ls
apiextensions-apiserver              kubeadm                    kube-controller-manager.docker_tag  kube-proxy.docker_tag      mounter
cloud-controller-manager             kube-apiserver             kube-controller-manager.tar         kube-proxy.tar
cloud-controller-manager.docker_tag  kube-apiserver.docker_tag  kubectl                             kube-scheduler
cloud-controller-manager.tar         kube-apiserver.tar         kubelet                             kube-scheduler.docker_tag
hyperkube                            kube-controller-manager    kube-proxy                          kube-scheduler.tar

5、将核心组件移动到制定位置

[root@master bin]# cp kube-apiserver kubectl kube-controller-manager kube-scheduler /opt/kubernetes/bin/

6、创建token令牌

[root@master bin]# cd /root/k8s
[root@master k8s]# head -c 16 /dev/urandom | od -An -t x | tr -d ' '	#生成随机序列号
5ff18641b25d7e18e62722ff4b119546

[root@master k8s]# vim /opt/kubernetes/cfg/token.csv
5ff18641b25d7e18e62722ff4b119546,kubelet-bootstrap,10001,"system:kubelet-bootstrap"

7、创建apiserver脚本生成apiserver的配置文件和启动脚本

[root@master k8s]# vim apiserver.sh
#!/bin/bash

MASTER_ADDRESS=$1
ETCD_SERVERS=$2

cat <<EOF >/opt/kubernetes/cfg/kube-apiserver

KUBE_APISERVER_OPTS="--logtostderr=true \\
--v=4 \\
--etcd-servers=${ETCD_SERVERS} \\
--bind-address=${MASTER_ADDRESS} \\
--secure-port=6443 \\
--advertise-address=${MASTER_ADDRESS} \\
--allow-privileged=true \\
--service-cluster-ip-range=10.0.0.0/24 \\
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \\
--authorization-mode=RBAC,Node \\
--kubelet-https=true \\
--enable-bootstrap-token-auth \\
--token-auth-file=/opt/kubernetes/cfg/token.csv \\
--service-node-port-range=30000-50000 \\
--tls-cert-file=/opt/kubernetes/ssl/server.pem  \\
--tls-private-key-file=/opt/kubernetes/ssl/server-key.pem \\
--client-ca-file=/opt/kubernetes/ssl/ca.pem \\
--service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \\
--etcd-cafile=/opt/etcd/ssl/ca.pem \\
--etcd-certfile=/opt/etcd/ssl/server.pem \\
--etcd-keyfile=/opt/etcd/ssl/server-key.pem"

EOF

cat <<EOF >/usr/lib/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes

[Service]
EnvironmentFile=-/opt/kubernetes/cfg/kube-apiserver
ExecStart=/opt/kubernetes/bin/kube-apiserver \$KUBE_APISERVER_OPTS
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable kube-apiserver
systemctl restart kube-apiserver

8、执行脚本生成apiserver的配置文件和启动文件

[root@master k8s]# bash apiserver.sh 20.0.0.10 https://20.0.0.10:2379,https://20.0.0.20:2379,https://20.0.0.30:2379

五、启动scheduler

1、创建scheduler脚本生成scheduler配置文件启动文件

[root@master k8s]# vim scheduler.sh
#!/bin/bash

MASTER_ADDRESS=$1

cat <<EOF >/opt/kubernetes/cfg/kube-scheduler

KUBE_SCHEDULER_OPTS="--logtostderr=true \\
--v=4 \\
--master=${MASTER_ADDRESS}:8080 \\
--leader-elect"

EOF

cat <<EOF >/usr/lib/systemd/system/kube-scheduler.service
[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/kubernetes/kubernetes

[Service]
EnvironmentFile=-/opt/kubernetes/cfg/kube-scheduler
ExecStart=/opt/kubernetes/bin/kube-scheduler \$KUBE_SCHEDULER_OPTS
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable kube-scheduler
systemctl restart kube-scheduler

2、执行脚本生成scheduler配置文件启动文件

[root@master k8s]# sh scheduler.sh 127.0.0.1

3、查看进程

[root@master k8s]#  ps aux | grep ku

六、启动controller-manager

1、创建controller-manager脚本生成controller-manager配置文件启动文件

[root@master k8s]# vim controller-manager
#!/bin/bash

MASTER_ADDRESS=$1

cat <<EOF >/opt/kubernetes/cfg/kube-controller-manager


KUBE_CONTROLLER_MANAGER_OPTS="--logtostderr=true \\
--v=4 \\
--master=${MASTER_ADDRESS}:8080 \\
--leader-elect=true \\
--address=127.0.0.1 \\
--service-cluster-ip-range=10.0.0.0/24 \\
--cluster-name=kubernetes \\
--cluster-signing-cert-file=/opt/kubernetes/ssl/ca.pem \\
--cluster-signing-key-file=/opt/kubernetes/ssl/ca-key.pem  \\
--root-ca-file=/opt/kubernetes/ssl/ca.pem \\
--service-account-private-key-file=/opt/kubernetes/ssl/ca-key.pem \\
--experimental-cluster-signing-duration=87600h0m0s"

EOF

cat <<EOF >/usr/lib/systemd/system/kube-controller-manager.service
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/kubernetes/kubernetes

[Service]
EnvironmentFile=-/opt/kubernetes/cfg/kube-controller-manager
ExecStart=/opt/kubernetes/bin/kube-controller-manager \$KUBE_CONTROLLER_MANAGER_OPTS
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable kube-controller-manager
systemctl restart kube-controller-manager

2、执行脚本生成scheduler配置文件启动文件

[root@master k8s]# sh controller-manager.sh 127.0.0.1
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-controller-manager.service to /usr/lib/systemd/system/kube-controller-manager.service.

3、查看进程

[root@master k8s]#  ps aux | grep ku

4、查看master节点状态

[root@master k8s]# /opt/kubernetes/bin/kubectl get cs
NAME                 STATUS    MESSAGE             ERROR
controller-manager   Healthy   ok                  
scheduler            Healthy   ok                  
etcd-1               Healthy   {"health":"true"}   
etcd-2               Healthy   {"health":"true"}   
etcd-0               Healthy   {"health":"true"}   
[root@master k8s]#

七、kubelet部署

master节点操作

1、拷贝kubelet kube-proxy工具到 node01和node02上

[root@master bin]# cd /root/k8s/kubernetes/server/bin
[root@master bin]# scp kubelet kube-proxy root@20.0.0.20:/opt/kubernetes/bin/
root@20.0.0.20's password: 
kubelet                                                                 100%  168MB 131.2MB/s   00:01    
kube-proxy                                                              100%   48MB  93.7MB/s   00:00    
[root@master bin]# scp kubelet kube-proxy root@20.0.0.30:/opt/kubernetes/bin/
root@20.0.0.30's password: 
kubelet                                                                 100%  168MB 139.0MB/s   00:01    
kube-proxy

2、master节点创建配置文件目录

[root@master bin]# cd /root/k8s
[root@master k8s]# mkdir kubeconfig
[root@master k8s]# cd kubeconfig/

3、查看token信息

[root@master kubeconfig]# cat /opt/kubernetes/cfg/token.csv 
5ff18641b25d7e18e62722ff4b119546,kubelet-bootstrap,10001,"system:kubelet-bootstrap"
#需要复制5ff18641b25d7e18e62722ff4b119546到后面创建的kubeconfig文件里

4、创建群集通信配置

[root@master kubeconfig]# vim kubeconfig.sh 
APISERVER=$1
SSL_DIR=$2

# 创建kubelet bootstrapping kubeconfig 
export KUBE_APISERVER="https://$APISERVER:6443"

# 设置集群参数
kubectl config set-cluster kubernetes \
  --certificate-authority=$SSL_DIR/ca.pem \
  --embed-certs=true \
  --server=${KUBE_APISERVER} \
  --kubeconfig=bootstrap.kubeconfig

# 设置客户端认证参数
kubectl config set-credentials kubelet-bootstrap \
  --token=5ff18641b25d7e18e62722ff4b119546 \
  --kubeconfig=bootstrap.kubeconfig

# 设置上下文参数
kubectl config set-context default \
  --cluster=kubernetes \
  --user=kubelet-bootstrap \
  --kubeconfig=bootstrap.kubeconfig

# 设置默认上下文
kubectl config use-context default --kubeconfig=bootstrap.kubeconfig

#----------------------

# 创建kube-proxy kubeconfig文件

kubectl config set-cluster kubernetes \
  --certificate-authority=$SSL_DIR/ca.pem \
  --embed-certs=true \
  --cluster=kubernetes \
  --user=kubelet-bootstrap \
  --kubeconfig=bootstrap.kubeconfig

# 设置默认上下文
kubectl config use-context default --kubeconfig=bootstrap.kubeconfig

#----------------------

# 创建kube-proxy kubeconfig文件

kubectl config set-cluster kubernetes \
  --certificate-authority=$SSL_DIR/ca.pem \
  --embed-certs=true \
  --kubeconfig=bootstrap.kubeconfig

# 设置上下文参数
kubectl config set-context default \
  --cluster=kubernetes \
  --user=kubelet-bootstrap \
  --kubeconfig=bootstrap.kubeconfig

# 设置默认上下文
kubectl config use-context default --kubeconfig=bootstrap.kubeconfig

#----------------------

# 创建kube-proxy kubeconfig文件

kubectl config set-cluster kubernetes \
  --certificate-authority=$SSL_DIR/ca.pem \
  --embed-certs=true \
  --server=${KUBE_APISERVER} \
  --kubeconfig=kube-proxy.kubeconfig

kubectl config set-credentials kube-proxy \
  --client-certificate=$SSL_DIR/kube-proxy.pem \
  --client-key=$SSL_DIR/kube-proxy-key.pem \
  --embed-certs=true \
  --kubeconfig=kube-proxy.kubeconfig

kubectl config set-context default \
  --cluster=kubernetes \
  --user=kube-proxy \
  --kubeconfig=kube-proxy.kubeconfig

kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig

5、重命名

[root@master kubeconfig]# mv kubeconfig.sh kubeconfig

6、将kubectl命令加入系统环境变量

[root@master kubeconfig]# vim /etc/profile
export PATH=$PATH:/opt/kubernetes/bin/	#在最后面增加

7、加载系统变量

[root@master kubeconfig]# source /etc/profile

8、查看maser节点组件是否正常

[root@master kubeconfig]# kubectl get cs
NAME                 STATUS    MESSAGE             ERROR
controller-manager   Healthy   ok                  
scheduler            Healthy   ok                  
etcd-0               Healthy   {"health":"true"}   
etcd-2               Healthy   {"health":"true"}   
etcd-1               Healthy   {"health":"true"}

9、执行群集通信配置

[root@master kubeconfig]# bash kubeconfig 20.0.0.10 /opt/kubernetes/ssl/
Cluster "kubernetes" set.
User "kubelet-bootstrap" set.
Context "default" created.
Switched to context "default".
Cluster "kubernetes" set.
User "kube-proxy" set.
Context "default" created.
Switched to context "default".

[root@master kubeconfig]# ls
bootstrap.kubeconfig  kubeconfig  kube-proxy.kubeconfig

10、将生成的配置文件拷贝到node01上面

[root@master kubeconfig]# scp bootstrap.kubeconfig kube-proxy.kubeconfig root@20.0.0.20:/opt/kubernetes/cfg/
root@20.0.0.20's password: 
bootstrap.kubeconfig                                                    100% 2163     1.4MB/s   00:00    
kube-proxy.kubeconfig                                                   100% 6269     4.3MB/s   00:00

11、master节点创建bootstrap角色赋予权限用于连接apiserver请求签名（关键）

[root@master kubeconfig]# kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --user=kubelet-bootstrap
clusterrolebinding.rbac.authorization.k8s.io/kubelet-bootstrap created

node节点
1、创建kubelet配置kubelet配置文件和启动文件

[root@node01 ~]# vim kubelet.sh
#!/bin/bash

NODE_ADDRESS=$1
DNS_SERVER_IP=${2:-"10.0.0.2"}

cat <<EOF >/opt/kubernetes/cfg/kubelet

KUBELET_OPTS="--logtostderr=true \\
--v=4 \\
--hostname-override=${NODE_ADDRESS} \\
--kubeconfig=/opt/kubernetes/cfg/kubelet.kubeconfig \\
--bootstrap-kubeconfig=/opt/kubernetes/cfg/bootstrap.kubeconfig \\
--config=/opt/kubernetes/cfg/kubelet.config \\
--cert-dir=/opt/kubernetes/ssl \\
--pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0"

EOF

cat <<EOF >/opt/kubernetes/cfg/kubelet.config

kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
address: ${NODE_ADDRESS}
port: 10250
readOnlyPort: 10255
cgroupDriver: cgroupfs
clusterDNS:
- ${DNS_SERVER_IP} 
clusterDomain: cluster.local.
failSwapOn: false
authentication:
  anonymous:
    enabled: true
EOF

cat <<EOF >/usr/lib/systemd/system/kubelet.service
[Unit]
Description=Kubernetes Kubelet
After=docker.service
Requires=docker.service

[Service]
EnvironmentFile=/opt/kubernetes/cfg/kubelet
ExecStart=/opt/kubernetes/bin/kubelet \$KUBELET_OPTS
Restart=on-failure
KillMode=process

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable kubelet
systemctl restart kubelet

2、执行脚本创建kubelet配置文件、启动文件

[root@node01 ~]# bash kubelet.sh 20.0.0.20
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service
#kubelet负责向master节点申请证书，启动时候就会去申请

3、master节点申请node01节点的申请

[root@master kubeconfig]# kubectl get csr	#查看节点申请
NAME                                                   AGE   REQUESTOR           CONDITION
node-csr-wiQUMex6280Dk5g5s-2mjWHEKF-EpVHcxtH4pv7cj6Y   2m    kubelet-bootstrap   Pending

[root@master kubeconfig]# kubectl certificate approve node-csr-wiQUMex6280Dk5g5s-2mjWHEKF-EpVHcxtH4pv7cj6Ycertificatesigningrequest.certificates.k8s.io/node-csr-wiQUMex6280Dk5g5s-2mjWHEKF-EpVHcxtH4pv7cj6Y approved

[root@master kubeconfig]# kubectl get csr
NAME                                                   AGE     REQUESTOR           CONDITION
node-csr-wiQUMex6280Dk5g5s-2mjWHEKF-EpVHcxtH4pv7cj6Y   4m56s   kubelet-bootstrap   Approved,Issued

八、kube-proxy部署

1、创建kube-proxy脚本创建kube-proxy配置文件和启动文件

[root@node01 ~]# vim proxy.sh
#!/bin/bash

NODE_ADDRESS=$1

cat <<EOF >/opt/kubernetes/cfg/kube-proxy

KUBE_PROXY_OPTS="--logtostderr=true \\
--v=4 \\
--hostname-override=${NODE_ADDRESS} \\
--cluster-cidr=10.0.0.0/24 \\
--proxy-mode=ipvs \\
--kubeconfig=/opt/kubernetes/cfg/kube-proxy.kubeconfig"

EOF

cat <<EOF >/usr/lib/systemd/system/kube-proxy.service
[Unit]
Description=Kubernetes Proxy
After=network.target

[Service]
EnvironmentFile=-/opt/kubernetes/cfg/kube-proxy
ExecStart=/opt/kubernetes/bin/kube-proxy \$KUBE_PROXY_OPTS
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable kube-proxy
systemctl restart kube-proxy

5、执行脚本

[root@node01 ~]# bash proxy.sh 20.0.0.20
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-proxy.service to /usr/lib/systemd/system/kube-proxy.service.

6、master节点查看node01节点是否加入

root@master kubeconfig]# kubectl get nodes
NAME        STATUS   ROLES    AGE     VERSION
20.0.0.20   Ready    <none>   3m56s   v1.12.3

7、将node02节点加入群集

#将node01节点的配置文件拷贝到node02节点上面
[root@node01 cfg]# scp -r /opt/kubernetes/ root@20.0.0.30:/opt/
root@20.0.0.30's password: 
flanneld                                                                100%  223   329.7KB/s   00:00    
bootstrap.kubeconfig                                                    100% 2163   380.7KB/s   00:00    
kube-proxy.kubeconfig                                                   100% 6269     8.2MB/s   00:00    
kubelet                                                                 100%  373   715.3KB/s   00:00    
kubelet.config                                                          100%  263   431.4KB/s   00:00    
kubelet.kubeconfig                                                      100% 2292     2.5MB/s   00:00    
kube-proxy                                                              100%  185   515.0KB/s   00:00    
scp: /opt//kubernetes/bin/flanneld: Text file busy
mk-docker-opts.sh                                                       100% 2139     7.7MB/s   00:00    
kubelet                                                                 100%  168MB 155.5MB/s   00:01    
kube-proxy                                                              100%   48MB 153.6MB/s   00:00    
kubelet.crt                                                             100% 2165   264.1KB/s   00:00    
kubelet.key                                                             100% 1679     2.5MB/s   00:00    
kubelet-client-2021-01-21-10-31-18.pem                                  100% 1269     2.6MB/s   00:00    
kubelet-client-current.pem                                              100% 1269     3.4MB/s   00:00    


#将node01节点的启动配置文件拷贝到node02节点上面
[root@node01 cfg]# scp /usr/lib/systemd/system/{kubelet,kube-proxy}.service root@20.0.0.30:/usr/lib/systemd/system/ 
root@20.0.0.30's password: 
kubelet.service                                                         100%  264   234.4KB/s   00:00    
kube-proxy.service

8、删除node01拷贝node02上的ssl证书

[root@node2 ~]# cd /opt/kubernetes/ssl/
[root@node2 ssl]# ls
kubelet-client-2021-01-21-10-31-18.pem  kubelet-client-current.pem  kubelet.crt  kubelet.key
[root@node2 ssl]# rm -fr *

9、修改kubelet IP地址指向自己

[root@node2 ssl]# vim /opt/kubernetes/cfg/kubelet
--hostname-override=20.0.0.30 \		#修改为自己主机IP地址

10、修改kubelet.config 的IP地址指向自己

[root@node2 ssl]# vim /opt/kubernetes/cfg/kubelet.config 
address: 20.0.0.30		#修改为自己主机IP地址

11、修改kube-proxy IP地址指向自己

[root@node2 ssl]# vim /opt/kubernetes/cfg/kube-proxy
--hostname-override=20.0.0.30 \		#修改为自己主机IP地址

12、启动kubelet和kube-proxy

[root@node2 ssl]# systemctl start kubelet.service
[root@node2 ssl]# systemctl enable kubelet.service
[root@node2 ssl]# systemctl start kube-proxy.service
[root@node2 ssl]# systemctl enable kube-proxy.service

13、master节点查看是否有申请证书的

[root@master kubeconfig]# kubectl get csr
NAME                                                   AGE     REQUESTOR           CONDITION
node-csr-4qcbVkszkvwHo0Ml4QK2wgV-VH3VHylecyGujC8d28M   2m29s   kubelet-bootstrap   Pending
node-csr-wiQUMex6280Dk5g5s-2mjWHEKF-EpVHcxtH4pv7cj6Y   53m     kubelet-bootstrap   Approved,Issued

#授权node02节点证书
[root@master kubeconfig]# kubectl certificate approve node-csr-4qcbVkszkvwHo0Ml4QK2wgV-VH3VHylecyGujC8d28M
certificatesigningrequest.certificates.k8s.io/node-csr-4qcbVkszkvwHo0Ml4QK2wgV-VH3VHylecyGujC8d28M approved

#查看证书是否授权成功
[root@master kubeconfig]# kubectl get csr
NAME                                                   AGE     REQUESTOR           CONDITION
node-csr-4qcbVkszkvwHo0Ml4QK2wgV-VH3VHylecyGujC8d28M   3m15s   kubelet-bootstrap   Approved,Issued
node-csr-wiQUMex6280Dk5g5s-2mjWHEKF-EpVHcxtH4pv7cj6Y   54m     kubelet-bootstrap   Approved,Issued

#查看节点是否加入成功
[root@master kubeconfig]# kubectl get nodes
NAME        STATUS   ROLES    AGE   VERSION
20.0.0.20   Ready    <none>   50m   v1.12.3
20.0.0.30   Ready    <none>   35s   v1.12.3

weixin_49226813

关注

0
点赞
踩
2

收藏

觉得还不错? 一键收藏
0
评论
k8s二进制单节点部署

目录部署规划一、etcd部署master节点部署node节点部署node节点加入群集部署规划名称主机名称IP地址master节点master20.0.0.10/24node节点node0120.0.0.20/24node节点node0220.0.0.30/24一、etcd部署master节点部署1、创建目录[root@master ~]# mkdir k8s[root@master ~]# cd k8s/[root@master k8s]#
复制链接

扫一扫

专栏目录