####kubernetes 特性
日我修复能力
弹性伸缩
自动部署和回滚
服务发现和负载均衡,服务发现是通过service 进行发现和负载均衡
机密和配置管理
存储编排
批处理
master节点:
kube-apiserver:集群的统一接口,各组件协调者,以restful api提供接口服务,所有对象资源的增删和监听操作都交给apiserver处理后在提交给etcd存储
kube-controller-manager:处理集群中常规后台服务,一个资源对应一个控制器,二controllermanager 就是负责管理这些控制器的
kube-scheduler:根据调度算法为新创建的pod 选择一个node 节点,可以任意部署,可以部署同一个节点上,也可以部署在不同的节点上。
etcd:分布式键值存储系统,用于保存集群状态数据,比如pod service 等存储信息
node节点
kubelet:kubelet是master在node节点上agent,管理本机运行容器的生命周期,比如创建容器,pod挂载数据卷,下载secret,获取容器和节点状态等工作,kubelet将每个pod转换成一组容器
kube-proxy:在node节点上实现pod网络代理,维护网络规则和四层负载均衡工作。
docker: 容器引擎,运行容器
###kubernetes 核心概念
pod:
。最小部署单元
。一组容器的集合
。一个pod中的容器共享网络命名空间
pod 是短暂的
controllers:
。replicaset:确保预期的pod副本数量
。deployment:无状态应用部署
。statefulset:有状态应用部署
。daemonset:确保所有的node运行同一个pod
。job:一致性服务
。crontab:定时任务
更高级层次对象,部署和管理pod
service:
。防止pod 失联
。定义一组pod的访问策略
label:
标签,附加到某个资源商,用于关联对象,查询和筛选
namespace:
命名空间,将对象逻辑上隔离
####二进制安装集群
1.生成集群证书,用cfssl 生成
curl -L https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -o /usr/local/bin/cfssl
curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /usr/local/bin/cfssljson
curl -L https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 -o /usr/local/bin/cfssl-certinfo
chmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson /usr/local/bin/cfssl-certinfo
2.集群证书列表
etcd: ca.pem,server.pem,server-key.pem
flannel: ca.pem,server.pem,server-key.pem
kube-apiserver:ca.pem,server.pem,server-key.pem
kubelet:ca.pem,ca-key.pem
kube-proxy: ca.pem,kube-proxy.pem,kube-proxy-key.pem
kubectl: ca.pem,admin,pem,admin-key.pem
3.
###etcd 安装,把etcd 需要生成的证书放到同一个文件下,写成一个脚本文件,执行
etcd:3.3.10
####生成证书
[root@master etcd]# cat etcd-cert.sh
cat > ca-config.json <<EOF
{
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"www": {
"expiry": "87600h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
}
}
}
}
EOF
cat > ca-csr.json <<EOF
{
"CN": "etcd CA",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "Beijing",
"ST": "Beijing"
}
]
}
EOF
cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
#-----------------------
cat > server-csr.json <<EOF
{
"CN": "etcd",
"hosts": [
"10.100.201.164",
"10.100.201.165",
"10.100.201.166"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "BeiJing",
"ST": "BeiJing"
}
]
}
EOF
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server
###解压包
tar -zxvf etcd-v3.3.10-linux-amd64.tar.gz /home/
mkdir -p /opt/etcd/{cfg,bin,ssl} -p
cp /home/etcd-v3.3.10-linux-amd64/{etcd,etcdctl} /opt/etcd/bin/
cp /home/etcd/{ca,server-key,server}.pem /opt/etcd/ssl/
###执行etcd.sh 生成配置文件和 服务启动文件
./etcd.sh etcd01 10.100.201.164 etcd02=https://10.100.201.165:2380,etcd03=https://10.100.201.166:2380
#!/bin/bash
# example: ./etcd.sh etcd01 10.100.201.164 etcd02=https://10.100.201.165:2380,etcd03=https://10.100.201.166:2380
ETCD_NAME=$1
ETCD_IP=$2
ETCD_CLUSTER=$3
WORK_DIR=/opt/etcd
cat <<EOF >$WORK_DIR/cfg/etcd
#[Member]
ETCD_NAME="${ETCD_NAME}"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://${ETCD_IP}:2380"
ETCD_LISTEN_CLIENT_URLS="https://${ETCD_IP}:2379"
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://${ETCD_IP}:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://${ETCD_IP}:2379"
ETCD_INITIAL_CLUSTER="etcd01=https://${ETCD_IP}:2380,${ETCD_CLUSTER}"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF
cat <<EOF >/usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
EnvironmentFile=${WORK_DIR}/cfg/etcd
ExecStart=${WORK_DIR}/bin/etcd \
--name=\${ETCD_NAME} \
--data-dir=\${ETCD_DATA_DIR} \
--listen-peer-urls=\${ETCD_LISTEN_PEER_URLS} \
--listen-client-urls=\${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 \
--advertise-client-urls=\${ETCD_ADVERTISE_CLIENT_URLS} \
--initial-advertise-peer-urls=\${ETCD_INITIAL_ADVERTISE_PEER_URLS} \
--initial-cluster=\${ETCD_INITIAL_CLUSTER} \
--initial-cluster-token=\${ETCD_INITIAL_CLUSTER_TOKEN} \
--initial-cluster-state=new \
--cert-file=${WORK_DIR}/ssl/server.pem \
--key-file=${WORK_DIR}/ssl/server-key.pem \
--peer-cert-file=${WORK_DIR}/ssl/server.pem \
--peer-key-file=${WORK_DIR}/ssl/server-key.pem \
--trusted-ca-file=${WORK_DIR}/ssl/ca.pem \
--peer-trusted-ca-file=${WORK_DIR}/ssl/ca.pem
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable etcd
systemctl start etcd
###测试 集群是否OK
/opt/etcd/bin/etcdctl --ca-file=/opt/etcd/ssl/ca.pem --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --endpoints="https://10.100.201.164:2379,https://10.100.201.165:2379,https://10.100.201.166:2379" cluster-health
member 2c1c1041aa163119 is healthy: got healthy result from https://10.100.201.166:2379
member 4764bce2dd9282dd is healthy: got healthy result from https://10.100.201.164:2379
member 8ebee0ef4b0805f7 is healthy: got healthy result from https://10.100.201.165:2379
cluster is healthy
######安装docker
wget -O /etc/yum.repos.d/Docker-ce.repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum install docker-ce-18* -y
##设置一个加速器
curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://f1361db2.m.daocloud.io
#### kubernetes 网络模型 CNI container network interface 容器网络接口
Kubernetes 网络模型设计基本要求:
。一个Pod 一个IP
。每个Pod独立有一个IP,Pod内所有容器共享网络(同一个IP)
。所有容器都可以与所有其他容器通信
。所有节点都可以与所有容器通信
Kubernetes 网络模型的实现 有哪些技术
隧道和路由方案
flannel :隧道 小规模,100以下
calico 路由 方案 BGP协议,网络策略, 大规模
weaveworks
ovs
contiv
romana
cilium
###安装 flannel 写入分配的子网到etcd 中,供flanneld使用 flannel网络的VXLAN及host-gw,这里用vxlan
/opt/etcd/bin/etcdctl --ca-file=/opt/etcd/ssl/ca.pem --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --endpoints="https://10.100.201.164:2379,https://10.100.201.165:2379,https://10.100.201.166:2379"
###
/opt/etcd/bin/etcdctl --ca-file=/opt/etcd/ssl/ca.pem --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --endpoints="https://10.100.201.164:2379,https://10.100.201.165:2379,https://10.100.201.166:2379" get /coreos.com/network/config '{"Netw^Ck":"171.16.0.0/16","Backend":{"Type":"vxlan"}}'
[root@master etcd]# /opt/etcd/bin/etcdctl --ca-file=/opt/etcd/ssl/ca.pem --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --endpoints="https://10.100.201.164:2379,https://10.100.201.165:2379,https://10.100.201.166:2379" list /coreos.com/network/config
No help topic for 'list'
[root@master etcd]# /opt/etcd/bin/etcdctl --ca-file=/opt/etcd/ssl/ca.pem --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --endpoints="https://10.100.201.164:2379,https://10.100.201.165:2379,https://10.100.201.166:2379" ls /coreos.com/network/config
###创建目录
mkdir /opt/kubernetes/{bin,cfg,ssl} -p
###解压 flannel-v0.10.0-linux-amd64.tar.gz
cp flannel-v0.10.0-linux-amd64/* /opt/kubernetes/bin/
#!/bin/bash
ETCD_ENDPOINTS=${1:-"http://127.0.0.1:2379"}
cat <<EOF >/opt/kubernetes/cfg/flanneld
FLANNEL_OPTIONS="--etcd-endpoints=${ETCD_ENDPOINTS} \
-etcd-cafile=/opt/etcd/ssl/ca.pem \
-etcd-certfile=/opt/etcd/ssl/server.pem \
-etcd-keyfile=/opt/etcd/ssl/server-key.pem"
EOF
cat <<EOF >/usr/lib/systemd/system/flanneld.service
[Unit]
Description=Flanneld overlay address etcd agent
After=network-online.target network.target
Before=docker.service
[Service]
Type=notify
EnvironmentFile=/opt/kubernetes/cfg/flanneld
ExecStart=/opt/kubernetes/bin/flanneld --ip-masq \$FLANNEL_OPTIONS
ExecStartPost=/opt/kubernetes/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/subnet.env
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF
cat <<EOF >/usr/lib/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service
Wants=network-online.target
[Service]
Type=notify
EnvironmentFile=/run/flannel/subnet.env
ExecStart=/usr/bin/dockerd \$DOCKER_NETWORK_OPTIONS
ExecReload=/bin/kill -s HUP \$MAINPID
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TimeoutStartSec=0
Delegate=yes
KillMode=process
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable flanneld
systemctl restart flanneld
systemctl restart docker
sh flannel.sh https://10.100.201.164:2379,https://10.100.201.165:2379,https://10.100.201.166:2379
scp /usr/lib/systemd/system/{docker,flanneld}.service root@10.100.201.166:/usr/lib/systemd/system/
####验证
docker 的网络变为和flannel 同一个段了
/opt/etcd/bin/etcdctl --endpoints="https://10.100.201.165:2379,https://10.100.201.164:2379.https://10.100.201.166:2379" --ca-file=/opt/etcd/ssl/ca.pem --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem get /coreos.com/network/subnets/171.16.95.0-24
{"PublicIP":"10.100.201.165","BackendType":"vxlan","BackendData":{"VtepMAC":"ca:7c:6c:28:83:5f"}}
####coredns
coredns:1.3.1
###kube-apiserver:v1.15.0
###上传这个 apiserver tar
tar -zxvf kubernetes-server-linux-amd64.tar.gz -C /home/k8s/
cp /home/k8s/kubernetes/server/bin/{kube-apiserver,kube-controller-manager,kube-scheduler} /opt/kubernetes/bin/
###生成证书 和token 文件
cat > ca-config.json <<EOF
{
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"kubernetes": {
"expiry": "87600h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
}
}
}
}
EOF
cat > ca-csr.json <<EOF
{
"CN": "kubernetes",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "Beijing",
"ST": "Beijing",
"O": "k8s",
"OU": "System"
}
]
}
EOF
cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
#-----------------------
cat > server-csr.json <<EOF
{
"CN": "kubernetes",
"hosts": [
"10.0.0.1",
"127.0.0.1",
"10.100.201.160",
"10.100.201.161",
"10.100.201.162",
"10.100.201.163",
"10.100.201.164",
"10.100.201.165",
"10.100.201.166",
"kubernetes",
"kubernetes.default",
"kubernetes.default.svc",
"kubernetes.default.svc.cluster",
"kubernetes.default.svc.cluster.local"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "BeiJing",
"ST": "BeiJing",
"O": "k8s",
"OU": "System"
}
]
}
EOF
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server
#-----------------------
cat > admin-csr.json <<EOF
{
"CN": "admin",
"hosts": [],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "BeiJing",
"ST": "BeiJing",
"O": "system:masters",
"OU": "System"
}
]
}
EOF
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin
#-----------------------
cat > kube-proxy-csr.json <<EOF
{
"CN": "system:kube-proxy",
"hosts": [],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "BeiJing",
"ST": "BeiJing",
"O": "k8s",
"OU": "System"
}
]
}
EOF
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy
chmod a+x k8s-cert.sh
cp server.pem server-key.pem ca.pem ca-key.pem /opt/kubernetes/ssl/
cat kubeconfig.sh
# 创建 TLS Bootstrapping Token
#BOOTSTRAP_TOKEN=$(head -c 16 /dev/urandom | od -An -t x | tr -d ' ')
BOOTSTRAP_TOKEN=0fb61c46f8991b718eb38d27b605b008
cat > token.csv <<EOF
${BOOTSTRAP_TOKEN},kubelet-bootstrap,10001,"system:kubelet-bootstrap"
EOF
#----------------------
APISERVER=$1
SSL_DIR=$2
# 创建kubelet bootstrapping kubeconfig
export KUBE_APISERVER="https://$APISERVER:6443"
# 设置集群参数
kubectl config set-cluster kubernetes \
--certificate-authority=$SSL_DIR/ca.pem \
--embed-certs=true \
--server=${KUBE_APISERVER} \
--kubeconfig=bootstrap.kubeconfig
# 设置客户端认证参数
kubectl config set-credentials kubelet-bootstrap \
--token=${BOOTSTRAP_TOKEN} \
--kubeconfig=bootstrap.kubeconfig
# 设置上下文参数
kubectl config set-context default \
--cluster=kubernetes \
--user=kubelet-bootstrap \
--kubeconfig=bootstrap.kubeconfig
# 设置默认上下文
kubectl config use-context default --kubeconfig=bootstrap.kubeconfig
#----------------------
# 创建kube-proxy kubeconfig文件
kubectl config set-cluster kubernetes \
--certificate-authority=$SSL_DIR/ca.pem \
--embed-certs=true \
--server=${KUBE_APISERVER} \
--kubeconfig=kube-proxy.kubeconfig
kubectl config set-credentials kube-proxy \
--client-certificate=$SSL_DIR/kube-proxy.pem \
--client-key=$SSL_DIR/kube-proxy-key.pem \
--embed-certs=true \
--kubeconfig=kube-proxy.kubeconfig
kubectl config set-context default \
--cluster=kubernetes \
--user=kube-proxy \
--kubeconfig=kube-proxy.kubeconfig
kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig
####生成token文件
BOOTSTRAP_TOKEN=0fb61c46f8991b718eb38d27b605b008
cat > token.csv <<EOF
${BOOTSTRAP_TOKEN},kubelet-bootstrap,10001,"system:kubelet-bootstrap"
EOF
###
[root@master k8s]# cat apiserver.sh
#!/bin/bash
MASTER_ADDRESS=$1
ETCD_SERVERS=$2
cat <<EOF >/opt/kubernetes/cfg/kube-apiserver
KUBE_APISERVER_OPTS="--logtostderr=true \\
--log-dir=/opt/kubernetes/logs \\
--v=4 \\
--etcd-servers=${ETCD_SERVERS} \\
--bind-address=${MASTER_ADDRESS} \\
--secure-port=6443 \\
--advertise-address=${MASTER_ADDRESS} \\
--allow-privileged=true \\
--service-cluster-ip-range=10.0.0.0/24 \\
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \\
--authorization-mode=RBAC,Node \\
--kubelet-https=true \\
--enable-bootstrap-token-auth \\
--token-auth-file=/opt/kubernetes/cfg/token.csv \\
--service-node-port-range=30000-50000 \\
--tls-cert-file=/opt/kubernetes/ssl/server.pem \\
--tls-private-key-file=/opt/kubernetes/ssl/server-key.pem \\
--client-ca-file=/opt/kubernetes/ssl/ca.pem \\
--service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \\
--etcd-cafile=/opt/etcd/ssl/ca.pem \\
--etcd-certfile=/opt/etcd/ssl/server.pem \\
--etcd-keyfile=/opt/etcd/ssl/server-key.pem"
EOF
cat <<EOF >/usr/lib/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes
[Service]
EnvironmentFile=-/opt/kubernetes/cfg/kube-apiserver
ExecStart=/opt/kubernetes/bin/kube-apiserver \$KUBE_APISERVER_OPTS
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable kube-apiserver
systemctl start kube-apiserver
###执行脚本
sh apiserver.sh 10.100.201.164 https://10.100.201.164:2379,https://10.100.201.165:2379,https://10.100.201.166:2379
###确认 服务状态
[root@master k8s]# cat controller-manager.sh
#!/bin/bash
MASTER_ADDRESS=$1
cat <<EOF >/opt/kubernetes/cfg/kube-controller-manager
KUBE_CONTROLLER_MANAGER_OPTS="--logtostderr=true \\
--log-dir=/opt/kubernetes/logs \\
--v=4 \\
--master=${MASTER_ADDRESS}:8080 \\
--leader-elect=true \\
--address=127.0.0.1 \\
--service-cluster-ip-range=10.0.0.0/24 \\
--cluster-name=kubernetes \\
--cluster-signing-cert-file=/opt/kubernetes/ssl/ca.pem \\
--cluster-signing-key-file=/opt/kubernetes/ssl/ca-key.pem \\
--root-ca-file=/opt/kubernetes/ssl/ca.pem \\
--service-account-private-key-file=/opt/kubernetes/ssl/ca-key.pem \\
--experimental-cluster-signing-duration=87600h0m0s"
EOF
cat <<EOF >/usr/lib/systemd/system/kube-controller-manager.service
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/kubernetes/kubernetes
[Service]
EnvironmentFile=-/opt/kubernetes/cfg/kube-controller-manager
ExecStart=/opt/kubernetes/bin/kube-controller-manager \$KUBE_CONTROLLER_MANAGER_OPTS
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable kube-controller-manager
systemctl start kube-controller-manager
sh controller-manager.sh 127.0.0.1
[root@master k8s]# cat scheduler.sh
#!/bin/bash
MASTER_ADDRESS=$1
cat <<EOF >/opt/kubernetes/cfg/kube-scheduler
KUBE_SCHEDULER_OPTS="--logtostderr=true \\
--v=4 \\
--master=${MASTER_ADDRESS}:8080 \\
--leader-elect"
EOF
cat <<EOF >/usr/lib/systemd/system/kube-scheduler.service
[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/kubernetes/kubernetes
[Service]
EnvironmentFile=-/opt/kubernetes/cfg/kube-scheduler
ExecStart=/opt/kubernetes/bin/kube-scheduler \$KUBE_SCHEDULER_OPTS
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable kube-scheduler
systemctl restart kube-scheduler
sh scheduler.sh 127.0.0.1
###确认一下 集群的状态
cp /home/k8s/kubernetes/server/bin/kubectl /usr/bin/
kubectl get cs
### 可以查看 命令 缩写 kubectl api-resources
#######安装node 节点
1. 将kubelet-bootstrap 用户绑定到系统集群角色
kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --user=kubelet-bootstrap
clusterrolebinding.rbac.authorization.k8s.io/kubelet-bootstrap created
2.创建kubeconfig 文件
[root@master k8s]# cat kubeconfig.sh
# 创建 TLS Bootstrapping Token
#BOOTSTRAP_TOKEN=$(head -c 16 /dev/urandom | od -An -t x | tr -d ' ')
#BOOTSTRAP_TOKEN=0fb61c46f8991b718eb38d27b605b008
#
#cat > token.csv <<EOF
#${BOOTSTRAP_TOKEN},kubelet-bootstrap,10001,"system:kubelet-bootstrap"
#EOF
#
#---------------------创建kubeconfig 文
APISERVER=$1
SSL_DIR=$2
# 创建kubelet bootstrapping kubeconfig
export KUBE_APISERVER="https://$APISERVER:6443"
# 设置集群参数
kubectl config set-cluster kubernetes \
--certificate-authority=$SSL_DIR/ca.pem \
--embed-certs=true \
--server=${KUBE_APISERVER} \
--kubeconfig=bootstrap.kubeconfig
# 设置客户端认证参数
kubectl config set-credentials kubelet-bootstrap \
--token=${BOOTSTRAP_TOKEN} \
--kubeconfig=bootstrap.kubeconfig
# 设置上下文参数
kubectl config set-context default \
--cluster=kubernetes \
--user=kubelet-bootstrap \
--kubeconfig=bootstrap.kubeconfig
# 设置默认上下文
kubectl config use-context default --kubeconfig=bootstrap.kubeconfig
#----------------------
# 创建kube-proxy kubeconfig文件
kubectl config set-cluster kubernetes \
--certificate-authority=$SSL_DIR/ca.pem \
--embed-certs=true \
--server=${KUBE_APISERVER} \
--kubeconfig=kube-proxy.kubeconfig
kubectl config set-credentials kube-proxy \
--client-certificate=$SSL_DIR/kube-proxy.pem \
--client-key=$SSL_DIR/kube-proxy-key.pem \
--embed-certs=true \
--kubeconfig=kube-proxy.kubeconfig
kubectl config set-context default \
--cluster=kubernetes \
--user=kube-proxy \
--kubeconfig=kube-proxy.kubeconfig
kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig
sh kubeconfig.sh 10.100.201.164 /home/k8s/
Cluster "kubernetes" set.
User "kubelet-bootstrap" set.
Context "default" modified.
Switched to context "default".
Cluster "kubernetes" set.
User "kube-proxy" set.
Context "default" modified.
Switched to context "default".
####生成这两个文件
[root@master k8s]# ll bootstrap.kubeconfig kube-proxy.kubeconfig
-rw------- 1 root root 2168 Oct 25 13:51 bootstrap.kubeconfig
-rw------- 1 root root 6270 Oct 25 13:46 kube-proxy.kubeconfig
###把这两个文件scp 到node节点,
3. 部署kubelet,kube-proxy 文件
scp /home/k8s/{bootstrap.kubeconfig,kube-proxy.kubeconfig} root@10.100.201.165:/home/k8s-node/
scp kubernetes/server/bin/{kubelet,kube-proxy} root@10.100.201.165:/home/k8s-node/
##另外还有2个配置文件
[root@node01 k8s-node]# cat kubelet.sh
#!/bin/bash
NODE_ADDRESS=$1
DNS_SERVER_IP=${2:-"10.0.0.2"}
cat <<EOF >/opt/kubernetes/cfg/kubelet
KUBELET_OPTS="--logtostderr=false \\
--log-dir=/opt/kubernetes/logs \\
--v=4 \\
--address=${NODE_ADDRESS} \\
--hostname-override=${NODE_ADDRESS} \\
--kubeconfig=/opt/kubernetes/cfg/kubelet.kubeconfig \\
--experimental-bootstrap-kubeconfig=/opt/kubernetes/cfg/bootstrap.kubeconfig \\
--config=/opt/kubernetes/cfg/kubelet.config \\
--cert-dir=/opt/kubernetes/ssl \\
--pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0"
EOF
cat <<EOF >/opt/kubernetes/cfg/kubelet.config
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
address: ${NODE_ADDRESS}
port: 10250
cgroupDriver: cgroupfs
clusterDNS:
- ${DNS_SERVER_IP}
clusterDomain: cluster.local.
failSwapOn: false
EOF
cat <<EOF >/usr/lib/systemd/system/kubelet.service
[Unit]
Description=Kubernetes Kubelet
After=docker.service
Requires=docker.service
[Service]
EnvironmentFile=/opt/kubernetes/cfg/kubelet
ExecStart=/opt/kubernetes/bin/kubelet \$KUBELET_OPTS
Restart=on-failure
KillMode=process
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable kubelet
systemctl start kubelet
###cp 文件
把相关文件cp 到 /opt/kubernetes/cfg和bin文件中
cp /home/k8s-node/{kubelet,kube-proxy} /opt/kubernetes/bin/
cp /home/k8s-node/{bootstrap.kubeconfig,kube-proxy.kubeconfig} /opt/kubernetes/cfg/
###执行脚本
sh kubelet.sh 10.100.201.165
##安装
[root@node01 k8s-node]# cat proxy.sh
#!/bin/bash
NODE_ADDRESS=$1
cat <<EOF >/opt/kubernetes/cfg/kube-proxy
KUBE_PROXY_OPTS="--logtostderr=false \\
--log-dir=/opt/kubernetes/logs \\
--v=4 \\
--hostname-override=${NODE_ADDRESS} \\
--cluster-cidr=10.0.0.0/24 \\
--proxy-mode=ipvs \\
--kubeconfig=/opt/kubernetes/cfg/kube-proxy.kubeconfig"
EOF
cat <<EOF >/usr/lib/systemd/system/kube-proxy.service
[Unit]
Description=Kubernetes Proxy
After=network.target
[Service]
EnvironmentFile=-/opt/kubernetes/cfg/kube-proxy
ExecStart=/opt/kubernetes/bin/kube-proxy \$KUBE_PROXY_OPTS
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable kube-proxy
systemctl restart kube-proxy
sh proxy.sh 10.100.201.165
###在master 上设置
[root@master ~]# kubectl get csr
NAME AGE REQUESTOR CONDITION
node-csr-Dr7bHIkUhBl6rjYTu02VDP_RqZc3IAmmIP5FSoar5fg 17m kubelet-bootstrap Pending
###颁发证书
[root@master ~]# kubectl certificate approve node-csr-Dr7bHIkUhBl6rjYTu02VDP_RqZc3IAmmIP5FSoar5fg
certificatesigningrequest.certificates.k8s.io/node-csr-Dr7bHIkUhBl6rjYTu02VDP_RqZc3IAmmIP5FSoar5fg approved
[root@master ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
10.100.201.165 Ready <none> 23s v1.12.1
[root@master ~]#
###扩容node 节点
scp -r root@10.100.201.165:/usr/lib/systemd/system/{kubelet,kube-proxy}.service /usr/lib/systemd/system/
scp -r root@10.100.201.165:/opt/kubernetes /opt/
###修改 一下配置配置文件
[root@node02 cfg]# grep -rin "165"
kubelet:5:--address=10.100.201.165 \
kubelet:6:--hostname-override=10.100.201.165 \
kubelet.config:4:address: 10.100.201.165
kube-proxy:5:--hostname-override=10.100.201.165 \
systemctl enable kubelet kube-proxy
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-proxy.service to /usr/lib/systemd/system/kube-proxy.service.
[root@node02 ~]# systemctl start kubelet kube-proxy
###在master 节点上操作
[root@master k8s]# kubectl get csr
NAME AGE REQUESTOR CONDITION
node-csr-Dr7bHIkUhBl6rjYTu02VDP_RqZc3IAmmIP5FSoar5fg 40m kubelet-bootstrap Approved,Issued
node-csr-PWc0r4DaRnUr_pkx53_PArwYBbjmgQKFnRQBpk1vsFQ 70s kubelet-bootstrap Pending
[root@master k8s]# kubectl certificate approve node-csr-PWc0r4DaRnUr_pkx53_PArwYBbjmgQKFnRQBpk1vsFQ
certificatesigningrequest.certificates.k8s.io/node-csr-PWc0r4DaRnUr_pkx53_PArwYBbjmgQKFnRQBpk1vsFQ approved
[root@master k8s]# kubectl get node
NAME STATUS ROLES AGE VERSION
10.100.201.165 Ready <none> 21m v1.12.1
10.100.201.166 Ready <none> 11s v1.12.
###