CC00030.CloudKubernetes——|KuberNetes&二进制部署.V08|5台Server|——|kubernetes-master|

一、kubernetes-master组件配置：kubernetes创建环境目录

### --- 为kubernetes组件创建环境目录
~~~		所有节点创建相关目录

[root@k8s-master01 ~]# mkdir -p /etc/kubernetes/manifests/ /etc/systemd/system/kubelet.service.d /var/lib/kubelet /var/log/kubernetes

二、kube-apiserver组件部署

### --- kube-apiserver

~~~     所有Master节点创建kube-apiserver service，
~~~     # 注意，如果不是高可用集群，192.168.1.20改为master01的地址

三、kube-apiserver配置文件

### --- k8s-master01配置文件创建

~~~     # 注意：本文档k8s service网段为10.96.0.0/12，
~~~     # 该网段不能和宿主机的网段、Pod网段:重复，按需修改

### --- 创建k8s-master01的kube-apiserver配置文件

[root@k8s-master01 ~]# cat /usr/lib/systemd/system/kube-apiserver.service 
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes
After=network.target

[Service]
ExecStart=/usr/local/bin/kube-apiserver \
      --v=2  \
      --logtostderr=true  \
      --allow-privileged=true  \
      --bind-address=0.0.0.0  \
      --secure-port=6443  \
      --insecure-port=0  \
      --advertise-address=192.168.1.11 \
      --service-cluster-ip-range=10.96.0.0/12  \
      --service-node-port-range=30000-32767  \
      --etcd-servers=https://192.168.1.11:2379,https://192.168.1.12:2379,https://192.168.1.13:2379 \
      --etcd-cafile=/etc/etcd/ssl/etcd-ca.pem  \
      --etcd-certfile=/etc/etcd/ssl/etcd.pem  \
      --etcd-keyfile=/etc/etcd/ssl/etcd-key.pem  \
      --client-ca-file=/etc/kubernetes/pki/ca.pem  \
      --tls-cert-file=/etc/kubernetes/pki/apiserver.pem  \
      --tls-private-key-file=/etc/kubernetes/pki/apiserver-key.pem  \
      --kubelet-client-certificate=/etc/kubernetes/pki/apiserver.pem  \
      --kubelet-client-key=/etc/kubernetes/pki/apiserver-key.pem  \
      --service-account-key-file=/etc/kubernetes/pki/sa.pub  \
      --service-account-signing-key-file=/etc/kubernetes/pki/sa.key  \
      --service-account-issuer=https://kubernetes.default.svc.cluster.local \
      --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname  \
      --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,ResourceQuota  \
      --authorization-mode=Node,RBAC  \
      --enable-bootstrap-token-auth=true  \
      --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.pem  \
      --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.pem  \
      --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client-key.pem  \
      --requestheader-allowed-names=aggregator  \
      --requestheader-group-headers=X-Remote-Group  \
      --requestheader-extra-headers-prefix=X-Remote-Extra-  \
      --requestheader-username-headers=X-Remote-User
      # --token-auth-file=/etc/kubernetes/token.csv

Restart=on-failure
RestartSec=10s
LimitNOFILE=65535

[Install]
WantedBy=multi-user.target

### --- k8s-master02配置文件创建

~~~     # 注意：本文档k8s service网段为10.96.0.0/12，
~~~     # 该网段不能和宿主机的网段、Pod网段:重复，按需修改

### --- 创建k8s-master02的kube-apiserver配置文件

[root@k8s-master02 ~]# cat /usr/lib/systemd/system/kube-apiserver.service 

[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes
After=network.target

[Service]
ExecStart=/usr/local/bin/kube-apiserver \
      --v=2  \
      --logtostderr=true  \
      --allow-privileged=true  \
      --bind-address=0.0.0.0  \
      --secure-port=6443  \
      --insecure-port=0  \
      --advertise-address=192.168.1.12 \
      --service-cluster-ip-range=10.96.0.0/12  \
      --service-node-port-range=30000-32767  \
      --etcd-servers=https://192.168.1.11:2379,https://192.168.1.12:2379,https://192.168.1.13:2379 \
      --etcd-cafile=/etc/etcd/ssl/etcd-ca.pem  \
      --etcd-certfile=/etc/etcd/ssl/etcd.pem  \
      --etcd-keyfile=/etc/etcd/ssl/etcd-key.pem  \
      --client-ca-file=/etc/kubernetes/pki/ca.pem  \
      --tls-cert-file=/etc/kubernetes/pki/apiserver.pem  \
      --tls-private-key-file=/etc/kubernetes/pki/apiserver-key.pem  \
      --kubelet-client-certificate=/etc/kubernetes/pki/apiserver.pem  \
      --kubelet-client-key=/etc/kubernetes/pki/apiserver-key.pem  \
      --service-account-key-file=/etc/kubernetes/pki/sa.pub  \
      --service-account-signing-key-file=/etc/kubernetes/pki/sa.key  \
      --service-account-issuer=https://kubernetes.default.svc.cluster.local \
      --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname  \
      --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,ResourceQuota  \
      --authorization-mode=Node,RBAC  \
      --enable-bootstrap-token-auth=true  \
      --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.pem  \
      --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.pem  \
      --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client-key.pem  \
      --requestheader-allowed-names=aggregator  \
      --requestheader-group-headers=X-Remote-Group  \
      --requestheader-extra-headers-prefix=X-Remote-Extra-  \
      --requestheader-username-headers=X-Remote-User
      # --token-auth-file=/etc/kubernetes/token.csv

Restart=on-failure
RestartSec=10s
LimitNOFILE=65535

[Install]
WantedBy=multi-user.target

### --- k8s-master03配置文件创建

~~~     # 注意：本文档k8s service网段为10.96.0.0/12，
~~~     # 该网段不能和宿主机的网段、Pod网段:重复，按需修改

### --- 创建k8s-master03的kube-apiserver配置文件

[root@k8s-master03 ~]# cat /usr/lib/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes
After=network.target

[Service]
ExecStart=/usr/local/bin/kube-apiserver \
      --v=2  \
      --logtostderr=true  \
      --allow-privileged=true  \
      --bind-address=0.0.0.0  \
      --secure-port=6443  \
      --insecure-port=0  \
      --advertise-address=192.168.1.13 \
      --service-cluster-ip-range=10.96.0.0/12  \
      --service-node-port-range=30000-32767  \
      --etcd-servers=https://192.168.1.11:2379,https://192.168.1.12:2379,https://192.168.1.13:2379 \
      --etcd-cafile=/etc/etcd/ssl/etcd-ca.pem  \
      --etcd-certfile=/etc/etcd/ssl/etcd.pem  \
      --etcd-keyfile=/etc/etcd/ssl/etcd-key.pem  \
      --client-ca-file=/etc/kubernetes/pki/ca.pem  \
      --tls-cert-file=/etc/kubernetes/pki/apiserver.pem  \
      --tls-private-key-file=/etc/kubernetes/pki/apiserver-key.pem  \
      --kubelet-client-certificate=/etc/kubernetes/pki/apiserver.pem  \
      --kubelet-client-key=/etc/kubernetes/pki/apiserver-key.pem  \
      --service-account-key-file=/etc/kubernetes/pki/sa.pub  \
      --service-account-signing-key-file=/etc/kubernetes/pki/sa.key  \
      --service-account-issuer=https://kubernetes.default.svc.cluster.local \
      --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname  \
      --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,ResourceQuota  \
      --authorization-mode=Node,RBAC  \
      --enable-bootstrap-token-auth=true  \
      --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.pem  \
      --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.pem  \
      --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client-key.pem  \
      --requestheader-allowed-names=aggregator  \
      --requestheader-group-headers=X-Remote-Group  \
      --requestheader-extra-headers-prefix=X-Remote-Extra-  \
      --requestheader-username-headers=X-Remote-User
      # --token-auth-file=/etc/kubernetes/token.csv

Restart=on-failure
RestartSec=10s
LimitNOFILE=65535

[Install]
WantedBy=multi-user.target

四、启动并查看状态

### --- 启动kube-apiserver
~~~     所有节点启动kube-apiserver服务

[root@k8s-master01 ~]# systemctl daemon-reload && systemctl enable --now kube-apiserver
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-apiserver.service to /usr/lib/systemd/system/kube-apiserver.service.

五、查看kube-apiserver状态

### --- 查看kube-apiserver状态
~~~     查看kube-apiserver状态

[root@k8s-master01 ~]# systemctl status kube-apiserver
   Active: active (running) since Wed 2021-05-12 20:31:44 CST; 9s ago
~~~     注：系统日志的这些提示可以忽略
May 12 20:32:18 k8s-master01 kube-apiserver[2665]: I0512 20:32:18.003891    2665 clientconn.go:948] ClientConn switching balancer to "pick_first"
May 12 20:32:18 k8s-master01 kube-apiserver[2665]: I0512 20:32:18.004322    2665 balancer_conn_wrappers.go:78] pickfirstBalancer: HandleSubConnStateChange: 0xc011c7c8a0, {CONNECTING <nil>}
May 12 20:32:18 k8s-master01 kube-apiserver[2665]: I0512 20:32:18.015201    2665 balancer_conn_wrappers.go:78] pickfirstBalancer: HandleSubConnStateChange: 0xc011c7c8a0, {READY <nil>}
May 12 20:32:18 k8s-master01 kube-apiserver[2665]: I0512 20:32:18.017047    2665 controlbuf.go:508] transport: loopyWriter.run returning. connection error: desc = "transport is closing"
May 12 20:32:19 k8s-master01 kube-apiserver[2665]: I0512 20:32:19.240254    2665 client.go:360] parsed scheme: "passthrough"
May 12 20:32:19 k8s-master01 kube-apiserver[2665]: I0512 20:32:19.240357    2665 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://192.168.1.11:2379  <nil> 0 <nil>}] <nil> <nil>}
May 12 20:32:19 k8s-master01 kube-apiserver[2665]: I0512 20:32:19.240382    2665 clientconn.go:948] ClientConn switching balancer to "pick_first"
May 12 20:32:19 k8s-master01 kube-apiserver[2665]: I0512 20:32:19.240769    2665 balancer_conn_wrappers.go:78] pickfirstBalancer: HandleSubConnStateChange: 0xc012273bf0, {CONNECTING <nil>}
May 12 20:32:19 k8s-master01 kube-apiserver[2665]: I0512 20:32:19.255310    2665 balancer_conn_wrappers.go:78] pickfirstBalancer: HandleSubConnStateChange: 0xc012273bf0, {READY <nil>}
May 12 20:32:19 k8s-master01 kube-apiserver[2665]: I0512 20:32:19.257151    2665 controlbuf.go:508] transport: loopyWriter.run returning. connection error: desc = "transport is closing