目标
使用 centos8 安装 kubernetes 1.24 版本
注意, kubernetes 1.24 已经不再支持 dockerd
软件版本
名称 | 版本 | 作用 |
---|---|---|
containerd.io | 1.6.15-3.1 | 容器 |
kubectl.x86_64 | 1.24.8-0 | k8s node 服务 |
kubelet.x86_64 | 1.24.8-0 | kubelet (k8s) 用户交还工具 |
kubeadm.x86_64 | 1.24.8-0 | 集群初始化工具 |
kubernetes-cni.x86_64 | 1.1.1-0 | cni 接口 |
所需镜像
所有镜像都需要存放在 k8s.io namespace 中
名称 | 版本 | 作用 |
---|---|---|
k8s.gcr.io/coredns/coredns | v1.8.6 | dns 解析 |
k8s.gcr.io/etcd | 3.5.5-0 | etcd 数据存储中心 |
k8s.gcr.io/kube-apiserver | v1.24.8 | api server api 接口 |
k8s.gcr.io/kube-controller-manager | v1.24.8 | controller manager 控制节点 |
k8s.gcr.io/kube-proxy | v1.24.8 | kube-proxy 网络 proxy |
k8s.gcr.io/kube-scheduler | v1.24.8 | kube-scheduler 调度器 |
k8s.gcr.io/pause | 3.7 | 镜像启动所需软件 |
registry.k8s.io/pause | 3.6 | 镜像启动所需软件 |
安装过程
系统包
yum -y install yum-utils device-mapper-persistent-data lvm2 libtool-ltdl libcgroup langpacks-en glibc-all-langpacks tc
contraind
yum install -y containerd.io.x86_64
contrainer 模块
echo -e "overlay\nbr_netfilter" > /etc/modules-load.d/containerd.conf
modprobe overlay
modprobe br_netfilter
修改 contrainer 配置
mkdir -p /etc/containerd
containerd config default | tee /etc/containerd/config.toml
sed -i 's/SystemdCgroup = false/SystemdCgroup = true/g' /etc/containerd/config.toml
如果需要在 contrainerd 中使用到 proxy
vim /usr/lib/systemd/system/containerd.service
[Service]
Environment="HTTP_PROXY=http://proxy_ip:port"
Environment="HTTPS_PROXY=http://proxy_ip:port"
Environment="NO_PROXY=.svr,127.0.0.1,节点1,节点2"
添加内核模块
echo -e "net.bridge.bridge-nf-call-iptables = 1\nnet.ipv4.ip_forward = 1\nnet.bridge.bridge-nf-call-ip6tables = 1" > /etc/sysctl.d/99-kubernetes-cri.conf
sysctl --system
selinux 关闭
setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
安装 kubelet
yum install -y kubelet kubeadm kubectl
镜像管理工具
https://github.com/kubernetes-sigs/cri-tools/releases/tag/v1.26.0
配置
cat /etc/crictl.yaml
runtime-endpoint: unix:///var/run/containerd/containerd.sock <- 使用 contrainred 作为 cri
image-endpoint: unix:///var/run/containerd/containerd.sock <- 使用 contrainerd 进行惊险管理
timeout: 2
debug: false
pull-image-on-create: false
disable-pull-on-run: false
master 配置
swapoff
sed -i /swap/d /etc/fstab
swapoff -a
检查需要的镜像版本
# kubeadm config images list
I0206 17:09:57.566434 40157 version.go:255] remote version is much newer: v1.26.1; falling back to: stable-1.24
k8s.gcr.io/kube-apiserver:v1.24.10
k8s.gcr.io/kube-controller-manager:v1.24.10
k8s.gcr.io/kube-scheduler:v1.24.10
k8s.gcr.io/kube-proxy:v1.24.10
k8s.gcr.io/pause:3.7
k8s.gcr.io/etcd:3.5.5-0
k8s.gcr.io/coredns/coredns:v1.8.6
安装详细日志过程
# kubeadm init --pod-network-cidr=10.244.0.0/16 --v=9
I0207 08:21:56.379975 46264 initconfiguration.go:117] detected and using CRI socket: unix:///var/run/containerd/containerd.sock
I0207 08:21:56.380241 46264 interface.go:432] Looking for default routes with IPv4 addresses
I0207 08:21:56.380274 46264 interface.go:437] Default route transits interface "bond0.20"
I0207 08:21:56.380733 46264 interface.go:209] Interface bond0.20 is up
I0207 08:21:56.380793 46264 interface.go:257] Interface "bond0.20" has 2 addresses :[10.189.20.64/24 fe80::ee38:8fff:fe79:26fe/64].
I0207 08:21:56.380829 46264 interface.go:224] Checking addr 10.189.20.64/24.
I0207 08:21:56.380843 46264 interface.go:231] IP found 10.189.20.64
I0207 08:21:56.380854 46264 interface.go:263] Found valid IPv4 address 10.189.20.64 for interface "bond0.20".
I0207 08:21:56.380865 46264 interface.go:443] Found active IP 10.189.20.64
I0207 08:21:56.380900 46264 kubelet.go:218] the value of KubeletConfiguration.cgroupDriver is empty; setting it to "systemd"
I0207 08:21:56.391103 46264 version.go:186] fetching Kubernetes version from URL: https://dl.k8s.io/release/stable-1.txt
W0207 08:21:56.624125 46264 version.go:103] could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt": Get "https://dl.k8s.io/release/stable-1.txt": x509: certificate signed by unknown authority
W0207 08:21:56.624154 46264 version.go:104] falling back to the local client version: v1.24.8
[init] Using Kubernetes version: v1.24.8
[preflight] Running pre-flight checks
I0207 08:21:56.624441 46264 checks.go:570] validating Kubernetes and kubeadm version
I0207 08:21:56.624468 46264 checks.go:170] validating if the firewall is enabled and active
I0207 08:21:56.645248 46264 checks.go:205] validating availability of port 6443
I0207 08:21:56.645424 46264 checks.go:205] validating availability of port 10259
I0207 08:21:56.645472 46264 checks.go:205] validating availability of port 10257
I0207 08:21:56.645511 46264 checks.go:282] validating the existence of file /etc/kubernetes/manifests/kube-apiserver.yaml
I0207 08:21:56.645539 46264 checks.go:282] validating the existence of file /etc/kubernetes/manifests/kube-controller-manager.yaml
I0207 08:21:56.645555 46264 checks.go:282] validating the existence of file /etc/kubernetes/manifests/kube-scheduler.yaml
I0207 08:21:56.645569 46264 checks.go:282] validating the existence of file /etc/kubernetes/manifests/etcd.yaml
I0207 08:21:56.645583 46264 checks.go:432] validating if the connectivity type is via proxy or direct
I0207 08:21:56.645605 46264 checks.go:471] validating http connectivity to first IP address in the CIDR
I0207 08:21:56.645627 46264 checks.go:471] validating http connectivity to first IP address in the CIDR
I0207 08:21:56.645654 46264 checks.go:106] validating the container runtime
I0207 08:21:56.673959 46264 checks.go:331] validating the contents of file /proc/sys/net/bridge/bridge-nf-call-iptables
I0207 08:21:56.674021 46264 checks.go:331] validating the contents of file /proc/sys/net/ipv4/ip_forward
I0207 08:21:56.674052 46264 checks.go:646] validating whether swap is enabled or not
I0207 08:21:56.674088 46264 checks.go:372] validating the presence of executable crictl
I0207 08:21:56.674113 46264 checks.go:372] validating the presence of executable conntrack
I0207 08:21:56.674130 46264 checks.go:372] validating the presence of executable ip
I0207 08:21:56.674146 46264 checks.go:372] validating the presence of executable iptables
I0207 08:21:56.674163 46264 checks.go:372] validating the presence of executable mount
I0207 08:21:56.674180 46264 checks.go:372] validating the presence of executable nsenter
I0207 08:21:56.674198 46264 checks.go:372] validating the presence of executable ebtables
I0207 08:21:56.674214 46264 checks.go:372] validating the presence of executable ethtool
I0207 08:21:56.674230 46264 checks.go:372] validating the presence of executable socat
I0207 08:21:56.674247 46264 checks.go:372] validating the presence of executable tc
I0207 08:21:56.674274 46264 checks.go:372] validating the presence of executable touch
I0207 08:21:56.674299 46264 checks.go:518] running all checks
I0207 08:21:56.681808 46264 checks.go:403] checking whether the given node name is valid and reachable using net.LookupHost
I0207 08:21:56.681838 46264 checks.go:612] validating kubelet version
I0207 08:21:56.737501 46264 checks.go:132] validating if the "kubelet" service is enabled and active
I0207 08:21:56.754844 46264 checks.go:205] validating availability of port 10250
I0207 08:21:56.754937 46264 checks.go:205] validating availability of port 2379
I0207 08:21:56.754981 46264 checks.go:205] validating availability of port 2380
I0207 08:21:56.755023 46264 checks.go:245] validating the existence and emptiness of directory /var/lib/etcd
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0207 08:21:56.755130 46264 checks.go:834] using image pull policy: IfNotPresent
I0207 08:21:56.782170 46264 checks.go:843] image exists: k8s.gcr.io/kube-apiserver:v1.24.8
I0207 08:21:56.808358 46264 checks.go:843] image exists: k8s.gcr.io/kube-controller-manager:v1.24.8
I0207 08:21:56.834233 46264 checks.go:843] image exists: k8s.gcr.io/kube-scheduler:v1.24.8
I0207 08:21:56.859979 46264 checks.go:843] image exists: k8s.gcr.io/kube-proxy:v1.24.8
I0207 08:21:56.885745 46264 checks.go:843] image exists: k8s.gcr.io/pause:3.7
I0207 08:21:56.911946 46264 checks.go:843] image exists: k8s.gcr.io/etcd:3.5.5-0
I0207 08:21:56.937651 46264 checks.go:843] image exists: k8s.gcr.io/coredns/coredns:v1.8.6
[certs] Using certificateDir folder "/etc/kubernetes/pki"
I0207 08:21:56.937724 46264 certs.go:112] creating a new certificate authority for ca
[certs] Generating "ca" certificate and key
I0207 08:21:57.071752 46264 certs.go:522] validating certificate period for ca certificate
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local ns-yun-020064.vclound.com] and IPs [10.96.0.1 10.189.20.64]
[certs] Generating "apiserver-kubelet-client" certificate and key
I0207 08:21:57.659884 46264 certs.go:112] creating a new certificate authority for front-proxy-ca
[certs] Generating "front-proxy-ca" certificate and key
I0207 08:21:57.794444 46264 certs.go:522] validating certificate period for front-proxy-ca certificate
[certs] Generating "front-proxy-client" certificate and key
I0207 08:21:57.965192 46264 certs.go:112] creating a new certificate authority for etcd-ca
[certs] Generating "etcd/ca" certificate and key
I0207 08:21:58.086396 46264 certs.go:522] validating certificate period for etcd/ca certificate
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost ns-yun-020064.vclound.com] and IPs [10.189.20.64 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost ns-yun-020064.vclound.com] and IPs [10.189.20.64 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
I0207 08:21:59.067506 46264 certs.go:78] creating new public/private key files for signing service account users
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0207 08:21:59.328104 46264 kubeconfig.go:103] creating kubeconfig file for admin.conf
[kubeconfig] Writing "admin.conf" kubeconfig file
I0207 08:21:59.455912 46264 kubeconfig.go:103] creating kubeconfig file for kubelet.conf
[kubeconfig] Writing "kubelet.conf" kubeconfig file
I0207 08:21:59.992365 46264 kubeconfig.go:103] creating kubeconfig file for controller-manager.conf
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0207 08:22:00.424942 46264 kubeconfig.go:103] creating kubeconfig file for scheduler.conf
[kubeconfig] Writing "scheduler.conf" kubeconfig file
I0207 08:22:00.646836 46264 kubelet.go:65] Stopping the kubelet
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
I0207 08:22:00.859398 46264 manifests.go:99] [control-plane] getting StaticPodSpecs
I0207 08:22:00.859759 46264 certs.go:522] validating certificate period for CA certificate
I0207 08:22:00.859872 46264 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-apiserver"
I0207 08:22:00.859889 46264 manifests.go:125] [control-plane] adding volume "etc-pki" for component "kube-apiserver"
I0207 08:22:00.859903 46264 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-apiserver"
I0207 08:22:00.863969 46264 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-apiserver" to "/etc/kubernetes/manifests/kube-apiserver.yaml"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
I0207 08:22:00.863995 46264 manifests.go:99] [control-plane] getting StaticPodSpecs
I0207 08:22:00.864249 46264 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-controller-manager"
I0207 08:22:00.864275 46264 manifests.go:125] [control-plane] adding volume "etc-pki" for component "kube-controller-manager"
I0207 08:22:00.864286 46264 manifests.go:125] [control-plane] adding volume "flexvolume-dir" for component "kube-controller-manager"
I0207 08:22:00.864295 46264 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-controller-manager"
I0207 08:22:00.864305 46264 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-controller-manager"
I0207 08:22:00.865129 46264 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-controller-manager" to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
[control-plane] Creating static Pod manifest for "kube-scheduler"
I0207 08:22:00.865149 46264 manifests.go:99] [control-plane] getting StaticPodSpecs
I0207 08:22:00.865393 46264 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-scheduler"
I0207 08:22:00.865952 46264 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-scheduler" to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0207 08:22:00.866794 46264 local.go:65] [etcd] wrote Static Pod manifest for a local etcd member to "/etc/kubernetes/manifests/etcd.yaml"
I0207 08:22:00.866811 46264 waitcontrolplane.go:83] [wait-control-plane] Waiting for the API server to be healthy
I0207 08:22:00.867582 46264 loader.go:372] Config loaded from file: /etc/kubernetes/admin.conf
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0207 08:22:00.868334 46264 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/healthz?timeout=10s'
I0207 08:22:00.868540 46264 round_trippers.go:508] HTTP Trace: Dial to tcp:10.189.20.64:6443 failed: dial tcp 10.189.20.64:6443: connect: connection refused
I0207 08:22:00.868568 46264 round_trippers.go:553] GET https://10.189.20.64:6443/healthz?timeout=10s in 0 milliseconds
I0207 08:22:00.868582 46264 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 0 ms Duration 0 ms
I0207 08:22:00.868594 46264 round_trippers.go:577] Response Headers:
I0207 08:22:01.369456 46264 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/healthz?timeout=10s'
I0207 08:22:01.369753 46264 round_trippers.go:508] HTTP Trace: Dial to tcp:10.189.20.64:6443 failed: dial tcp 10.189.20.64:6443: connect: connection refused
I0207 08:22:01.369794 46264 round_trippers.go:553] GET https://10.189.20.64:6443/healthz?timeout=10s in 0 milliseconds
I0207 08:22:01.369813 46264 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 0 ms Duration 0 ms
I0207 08:22:01.369834 46264 round_trippers.go:577] Response Headers:
I0207 08:22:01.869586 46264 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/healthz?timeout=10s'
I0207 08:22:01.869932 46264 round_trippers.go:508] HTTP Trace: Dial to tcp:10.189.20.64:6443 failed: dial tcp 10.189.20.64:6443: connect: connection refused
I0207 08:22:01.869970 46264 round_trippers.go:553] GET https://10.189.20.64:6443/healthz?timeout=10s in 0 milliseconds
I0207 08:22:01.869994 46264 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 0 ms Duration 0 ms
I0207 08:22:01.870010 46264 round_trippers.go:577] Response Headers:
I0207 08:22:02.369678 46264 round_trippers.go:466] curl -v -XGET -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" -H "Accept: application/json, */*" 'https://10.189.20.64:6443/healthz?timeout=10s'
I0207 08:22:02.370100 46264 round_trippers.go:508] HTTP Trace: Dial to tcp:10.189.20.64:6443 failed: dial tcp 10.189.20.64:6443: connect: connection refused
I0207 08:22:02.370142 46264 round_trippers.go:553] GET https://10.189.20.64:6443/healthz?timeout=10s in 0 milliseconds
I0207 08:22:02.370168 46264 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 0 ms Duration 0 ms
I0207 08:22:02.370188 46264 round_trippers.go:577] Response Headers:
I0207 08:22:02.869837 46264 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/healthz?timeout=10s'
I0207 08:22:02.870127 46264 round_trippers.go:510] HTTP Trace: Dial to tcp:10.189.20.64:6443 succeed
I0207 08:22:06.099994 46264 round_trippers.go:553] GET https://10.189.20.64:6443/healthz?timeout=10s 500 Internal Server Error in 3230 milliseconds
I0207 08:22:06.100044 46264 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 3225 ms ServerProcessing 3 ms Duration 3230 ms
I0207 08:22:06.100062 46264 round_trippers.go:577] Response Headers:
I0207 08:22:06.100080 46264 round_trippers.go:580] X-Kubernetes-Pf-Prioritylevel-Uid:
I0207 08:22:06.100096 46264 round_trippers.go:580] Content-Length: 1263
I0207 08:22:06.100112 46264 round_trippers.go:580] Date: Tue, 07 Feb 2023 00:22:06 GMT
I0207 08:22:06.100127 46264 round_trippers.go:580] Audit-Id: 70822272-9f2a-4c44-97c9-6b816800ee08
I0207 08:22:06.100143 46264 round_trippers.go:580] Cache-Control: no-cache, private
I0207 08:22:06.100159 46264 round_trippers.go:580] Content-Type: text/plain; charset=utf-8
I0207 08:22:06.100174 46264 round_trippers.go:580] X-Content-Type-Options: nosniff
I0207 08:22:06.100189 46264 round_trippers.go:580] X-Kubernetes-Pf-Flowschema-Uid:
I0207 08:22:06.100255 46264 request.go:1154] Response Body: [+]ping ok
[+]log ok
[+]etcd ok
[+]poststarthook/start-kube-apiserver-admission-initializer ok
[+]poststarthook/generic-apiserver-start-informers ok
[+]poststarthook/priority-and-fairness-config-consumer ok
[+]poststarthook/priority-and-fairness-filter ok
[+]poststarthook/start-apiextensions-informers ok
[-]poststarthook/start-apiextensions-controllers failed: reason withheld
[-]poststarthook/crd-informer-synced failed: reason withheld
[-]poststarthook/bootstrap-controller failed: reason withheld
[-]poststarthook/rbac/bootstrap-roles failed: reason withheld
[-]poststarthook/scheduling/bootstrap-system-priority-classes failed: reason withheld
[-]poststarthook/priority-and-fairness-config-producer failed: reason withheld
[+]poststarthook/start-cluster-authentication-info-controller ok
[+]poststarthook/aggregator-reload-proxy-client-cert ok
[+]poststarthook/start-kube-aggregator-informers ok
[-]poststarthook/apiservice-registration-controller failed: reason withheld
[+]poststarthook/apiservice-status-available-controller ok
[+]poststarthook/kube-apiserver-autoregistration ok
[-]autoregister-completion failed: reason withheld
[+]poststarthook/apiservice-openapi-controller ok
[+]poststarthook/apiservice-openapiv3-controller ok
healthz check failed
I0207 08:22:06.368953 46264 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/healthz?timeout=10s'
I0207 08:22:06.370374 46264 round_trippers.go:553] GET https://10.189.20.64:6443/healthz?timeout=10s 500 Internal Server Error in 1 milliseconds
I0207 08:22:06.370396 46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:06.370414 46264 round_trippers.go:577] Response Headers:
I0207 08:22:06.370427 46264 round_trippers.go:580] Cache-Control: no-cache, private
I0207 08:22:06.370440 46264 round_trippers.go:580] Content-Type: text/plain; charset=utf-8
I0207 08:22:06.370450 46264 round_trippers.go:580] X-Content-Type-Options: nosniff
I0207 08:22:06.370461 46264 round_trippers.go:580] X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:06.370471 46264 round_trippers.go:580] X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:06.370483 46264 round_trippers.go:580] Content-Length: 1137
I0207 08:22:06.370493 46264 round_trippers.go:580] Date: Tue, 07 Feb 2023 00:22:06 GMT
I0207 08:22:06.370504 46264 round_trippers.go:580] Audit-Id: c360ea9a-d67c-4017-8327-f1951446880b
I0207 08:22:06.370542 46264 request.go:1154] Response Body: [+]ping ok
[+]log ok
[+]etcd ok
[+]poststarthook/start-kube-apiserver-admission-initializer ok
[+]poststarthook/generic-apiserver-start-informers ok
[+]poststarthook/priority-and-fairness-config-consumer ok
[+]poststarthook/priority-and-fairness-filter ok
[+]poststarthook/start-apiextensions-informers ok
[+]poststarthook/start-apiextensions-controllers ok
[+]poststarthook/crd-informer-synced ok
[+]poststarthook/bootstrap-controller ok
[-]poststarthook/rbac/bootstrap-roles failed: reason withheld
[-]poststarthook/scheduling/bootstrap-system-priority-classes failed: reason withheld
[+]poststarthook/priority-and-fairness-config-producer ok
[+]poststarthook/start-cluster-authentication-info-controller ok
[+]poststarthook/aggregator-reload-proxy-client-cert ok
[+]poststarthook/start-kube-aggregator-informers ok
[+]poststarthook/apiservice-registration-controller ok
[+]poststarthook/apiservice-status-available-controller ok
[+]poststarthook/kube-apiserver-autoregistration ok
[+]autoregister-completion ok
[+]poststarthook/apiservice-openapi-controller ok
[+]poststarthook/apiservice-openapiv3-controller ok
healthz check failed
I0207 08:22:06.869063 46264 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/healthz?timeout=10s'
I0207 08:22:06.870317 46264 round_trippers.go:553] GET https://10.189.20.64:6443/healthz?timeout=10s 500 Internal Server Error in 1 milliseconds
I0207 08:22:06.870339 46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:06.870352 46264 round_trippers.go:577] Response Headers:
I0207 08:22:06.870364 46264 round_trippers.go:580] X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:06.870376 46264 round_trippers.go:580] X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:06.870387 46264 round_trippers.go:580] Content-Length: 1137
I0207 08:22:06.870397 46264 round_trippers.go:580] Date: Tue, 07 Feb 2023 00:22:06 GMT
I0207 08:22:06.870408 46264 round_trippers.go:580] Audit-Id: 8b0409a8-1fd4-4247-ad17-a8ee0b50828c
I0207 08:22:06.870419 46264 round_trippers.go:580] Cache-Control: no-cache, private
I0207 08:22:06.870429 46264 round_trippers.go:580] Content-Type: text/plain; charset=utf-8
I0207 08:22:06.870440 46264 round_trippers.go:580] X-Content-Type-Options: nosniff
I0207 08:22:06.870468 46264 request.go:1154] Response Body: [+]ping ok
[+]log ok
[+]etcd ok
[+]poststarthook/start-kube-apiserver-admission-initializer ok
[+]poststarthook/generic-apiserver-start-informers ok
[+]poststarthook/priority-and-fairness-config-consumer ok
[+]poststarthook/priority-and-fairness-filter ok
[+]poststarthook/start-apiextensions-informers ok
[+]poststarthook/start-apiextensions-controllers ok
[+]poststarthook/crd-informer-synced ok
[+]poststarthook/bootstrap-controller ok
[-]poststarthook/rbac/bootstrap-roles failed: reason withheld
[-]poststarthook/scheduling/bootstrap-system-priority-classes failed: reason withheld
[+]poststarthook/priority-and-fairness-config-producer ok
[+]poststarthook/start-cluster-authentication-info-controller ok
[+]poststarthook/aggregator-reload-proxy-client-cert ok
[+]poststarthook/start-kube-aggregator-informers ok
[+]poststarthook/apiservice-registration-controller ok
[+]poststarthook/apiservice-status-available-controller ok
[+]poststarthook/kube-apiserver-autoregistration ok
[+]autoregister-completion ok
[+]poststarthook/apiservice-openapi-controller ok
[+]poststarthook/apiservice-openapiv3-controller ok
healthz check failed
I0207 08:22:07.369063 46264 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/healthz?timeout=10s'
I0207 08:22:07.370350 46264 round_trippers.go:553] GET https://10.189.20.64:6443/healthz?timeout=10s 500 Internal Server Error in 1 milliseconds
I0207 08:22:07.370377 46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:07.370389 46264 round_trippers.go:577] Response Headers:
I0207 08:22:07.370402 46264 round_trippers.go:580] Cache-Control: no-cache, private
I0207 08:22:07.370413 46264 round_trippers.go:580] Content-Type: text/plain; charset=utf-8
I0207 08:22:07.370423 46264 round_trippers.go:580] X-Content-Type-Options: nosniff
I0207 08:22:07.370434 46264 round_trippers.go:580] X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:07.370445 46264 round_trippers.go:580] X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:07.370456 46264 round_trippers.go:580] Content-Length: 1116
I0207 08:22:07.370466 46264 round_trippers.go:580] Date: Tue, 07 Feb 2023 00:22:07 GMT
I0207 08:22:07.370477 46264 round_trippers.go:580] Audit-Id: f2718e1e-acd1-4896-8b48-cf23392703e1
I0207 08:22:07.370514 46264 request.go:1154] Response Body: [+]ping ok
[+]log ok
[+]etcd ok
[+]poststarthook/start-kube-apiserver-admission-initializer ok
[+]poststarthook/generic-apiserver-start-informers ok
[+]poststarthook/priority-and-fairness-config-consumer ok
[+]poststarthook/priority-and-fairness-filter ok
[+]poststarthook/start-apiextensions-informers ok
[+]poststarthook/start-apiextensions-controllers ok
[+]poststarthook/crd-informer-synced ok
[+]poststarthook/bootstrap-controller ok
[-]poststarthook/rbac/bootstrap-roles failed: reason withheld
[+]poststarthook/scheduling/bootstrap-system-priority-classes ok
[+]poststarthook/priority-and-fairness-config-producer ok
[+]poststarthook/start-cluster-authentication-info-controller ok
[+]poststarthook/aggregator-reload-proxy-client-cert ok
[+]poststarthook/start-kube-aggregator-informers ok
[+]poststarthook/apiservice-registration-controller ok
[+]poststarthook/apiservice-status-available-controller ok
[+]poststarthook/kube-apiserver-autoregistration ok
[+]autoregister-completion ok
[+]poststarthook/apiservice-openapi-controller ok
[+]poststarthook/apiservice-openapiv3-controller ok
healthz check failed
I0207 08:22:07.869107 46264 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/healthz?timeout=10s'
I0207 08:22:07.870718 46264 round_trippers.go:553] GET https://10.189.20.64:6443/healthz?timeout=10s 200 OK in 1 milliseconds
I0207 08:22:07.870740 46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:07.870752 46264 round_trippers.go:577] Response Headers:
I0207 08:22:07.870765 46264 round_trippers.go:580] Cache-Control: no-cache, private
I0207 08:22:07.870776 46264 round_trippers.go:580] Content-Type: text/plain; charset=utf-8
I0207 08:22:07.870787 46264 round_trippers.go:580] X-Content-Type-Options: nosniff
I0207 08:22:07.870797 46264 round_trippers.go:580] X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:07.870808 46264 round_trippers.go:580] X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:07.870818 46264 round_trippers.go:580] Content-Length: 2
I0207 08:22:07.870829 46264 round_trippers.go:580] Date: Tue, 07 Feb 2023 00:22:07 GMT
I0207 08:22:07.870840 46264 round_trippers.go:580] Audit-Id: bc001798-04ca-4a38-8a77-8e36c065f40d
I0207 08:22:07.870885 46264 request.go:1154] Response Body: ok
[apiclient] All control plane components are healthy after 7.002728 seconds
I0207 08:22:07.870954 46264 uploadconfig.go:110] [upload-config] Uploading the kubeadm ClusterConfiguration to a ConfigMap
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
I0207 08:22:07.871630 46264 request.go:1154] Request Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"kubeadm-config","namespace":"kube-system","creationTimestamp":null},"data":{"ClusterConfiguration":"apiServer:\n extraArgs:\n authorization-mode: Node,RBAC\n timeoutForControlPlane: 4m0s\napiVersion: kubeadm.k8s.io/v1beta3\ncertificatesDir: /etc/kubernetes/pki\nclusterName: kubernetes\ncontrollerManager: {}\ndns: {}\netcd:\n local:\n dataDir: /var/lib/etcd\nimageRepository: k8s.gcr.io\nkind: ClusterConfiguration\nkubernetesVersion: v1.24.8\nnetworking:\n dnsDomain: cluster.local\n podSubnet: 10.244.0.0/16\n serviceSubnet: 10.96.0.0/12\nscheduler: {}\n"}}
I0207 08:22:07.871703 46264 round_trippers.go:466] curl -v -XPOST -H "Content-Type: application/json" -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s'
I0207 08:22:07.873785 46264 round_trippers.go:553] POST https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 2 milliseconds
I0207 08:22:07.873807 46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 2 ms
I0207 08:22:07.873823 46264 round_trippers.go:577] Response Headers:
I0207 08:22:07.873835 46264 round_trippers.go:580] Audit-Id: fc5476cb-f980-4faa-bb09-28fd8c5298cd
I0207 08:22:07.873846 46264 round_trippers.go:580] Cache-Control: no-cache, private
I0207 08:22:07.873857 46264 round_trippers.go:580] Content-Type: application/json
I0207 08:22:07.873867 46264 round_trippers.go:580] X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:07.873878 46264 round_trippers.go:580] X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:07.873889 46264 round_trippers.go:580] Content-Length: 910
I0207 08:22:07.873899 46264 round_trippers.go:580] Date: Tue, 07 Feb 2023 00:22:07 GMT
I0207 08:22:07.873928 46264 request.go:1154] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"kubeadm-config","namespace":"kube-system","uid":"ea94739b-1e8c-4a2f-ad2b-0b9c1769816d","resourceVersion":"204","creationTimestamp":"2023-02-07T00:22:07Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:07Z","fieldsType":"FieldsV1","fieldsV1":{"f:data":{".":{},"f:ClusterConfiguration":{}}}}]},"data":{"ClusterConfiguration":"apiServer:\n extraArgs:\n authorization-mode: Node,RBAC\n timeoutForControlPlane: 4m0s\napiVersion: kubeadm.k8s.io/v1beta3\ncertificatesDir: /etc/kubernetes/pki\nclusterName: kubernetes\ncontrollerManager: {}\ndns: {}\netcd:\n local:\n dataDir: /var/lib/etcd\nimageRepository: k8s.gcr.io\nkind: ClusterConfiguration\nkubernetesVersion: v1.24.8\nnetworking:\n dnsDomain: cluster.local\n podSubnet: 10.244.0.0/16\n serviceSubnet: 10.96.0.0/12\nscheduler: {}\n"}}
I0207 08:22:07.874313 46264 request.go:1154] Request Body: {"kind":"Role","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:nodes-kubeadm-config","namespace":"kube-system","creationTimestamp":null},"rules":[{"verbs":["get"],"apiGroups":[""],"resources":["configmaps"],"resourceNames":["kubeadm-config"]}]}
I0207 08:22:07.874385 46264 round_trippers.go:466] curl -v -XPOST -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" -H "Accept: application/json, */*" -H "Content-Type: application/json" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s'
I0207 08:22:07.876285 46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 201 Created in 1 milliseconds
I0207 08:22:07.876305 46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:07.876316 46264 round_trippers.go:577] Response Headers:
I0207 08:22:07.876327 46264 round_trippers.go:580] Content-Type: application/json
I0207 08:22:07.876338 46264 round_trippers.go:580] X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:07.876349 46264 round_trippers.go:580] X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:07.876360 46264 round_trippers.go:580] Content-Length: 541
I0207 08:22:07.876370 46264 round_trippers.go:580] Date: Tue, 07 Feb 2023 00:22:07 GMT
I0207 08:22:07.876381 46264 round_trippers.go:580] Audit-Id: 6756eb4f-54d6-4045-8d90-a0611df7d9ad
I0207 08:22:07.876391 46264 round_trippers.go:580] Cache-Control: no-cache, private
I0207 08:22:07.876413 46264 request.go:1154] Response Body: {"kind":"Role","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:nodes-kubeadm-config","namespace":"kube-system","uid":"0956bc31-f01d-43b1-af7c-0d94e93a6da9","resourceVersion":"205","creationTimestamp":"2023-02-07T00:22:07Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2023-02-07T00:22:07Z","fieldsType":"FieldsV1","fieldsV1":{"f:rules":{}}}]},"rules":[{"verbs":["get"],"apiGroups":[""],"resources":["configmaps"],"resourceNames":["kubeadm-config"]}]}
I0207 08:22:07.876649 46264 request.go:1154] Request Body: {"kind":"RoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:nodes-kubeadm-config","namespace":"kube-system","creationTimestamp":null},"subjects":[{"kind":"Group","name":"system:bootstrappers:kubeadm:default-node-token"},{"kind":"Group","name":"system:nodes"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"Role","name":"kubeadm:nodes-kubeadm-config"}}
I0207 08:22:07.876715 46264 round_trippers.go:466] curl -v -XPOST -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s'
I0207 08:22:07.878732 46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 201 Created in 1 milliseconds
I0207 08:22:07.878753 46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:07.878765 46264 round_trippers.go:577] Response Headers:
I0207 08:22:07.878777 46264 round_trippers.go:580] Audit-Id: 6eeda9a6-693d-45a1-9095-1295d6a2794b
I0207 08:22:07.878788 46264 round_trippers.go:580] Cache-Control: no-cache, private
I0207 08:22:07.878798 46264 round_trippers.go:580] Content-Type: application/json
I0207 08:22:07.878810 46264 round_trippers.go:580] X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:07.878821 46264 round_trippers.go:580] X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:07.878832 46264 round_trippers.go:580] Content-Length: 766
I0207 08:22:07.878843 46264 round_trippers.go:580] Date: Tue, 07 Feb 2023 00:22:07 GMT
I0207 08:22:07.878869 46264 request.go:1154] Response Body: {"kind":"RoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:nodes-kubeadm-config","namespace":"kube-system","uid":"c4cb7f9b-0235-405b-b521-d318a4012638","resourceVersion":"206","creationTimestamp":"2023-02-07T00:22:07Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2023-02-07T00:22:07Z","fieldsType":"FieldsV1","fieldsV1":{"f:roleRef":{},"f:subjects":{}}}]},"subjects":[{"kind":"Group","apiGroup":"rbac.authorization.k8s.io","name":"system:bootstrappers:kubeadm:default-node-token"},{"kind":"Group","apiGroup":"rbac.authorization.k8s.io","name":"system:nodes"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"Role","name":"kubeadm:nodes-kubeadm-config"}}
I0207 08:22:07.879035 46264 uploadconfig.go:124] [upload-config] Uploading the kubelet component config to a ConfigMap
[kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
I0207 08:22:07.879585 46264 request.go:1154] Request Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"kubelet-config","namespace":"kube-system","creationTimestamp":null,"annotations":{"kubeadm.kubernetes.io/component-config.hash":"sha256:53821686a8ddaca8403df63803a726d2ab31b145c7da69ed7b78c45d2e223a18"}},"data":{"kubelet":"apiVersion: kubelet.config.k8s.io/v1beta1\nauthentication:\n anonymous:\n enabled: false\n webhook:\n cacheTTL: 0s\n enabled: true\n x509:\n clientCAFile: /etc/kubernetes/pki/ca.crt\nauthorization:\n mode: Webhook\n webhook:\n cacheAuthorizedTTL: 0s\n cacheUnauthorizedTTL: 0s\ncgroupDriver: systemd\nclusterDNS:\n- 10.96.0.10\nclusterDomain: cluster.local\ncpuManagerReconcilePeriod: 0s\nevictionPressureTransitionPeriod: 0s\nfileCheckFrequency: 0s\nhealthzBindAddress: 127.0.0.1\nhealthzPort: 10248\nhttpCheckFrequency: 0s\nimageMinimumGCAge: 0s\nkind: KubeletConfiguration\nlogging:\n flushFrequency: 0\n options:\n json:\n infoBufferSize: \"0\"\n verbosity: 0\nmemorySwap: {}\nnodeStatusReportFrequency: 0s\nnodeStatusUpdateFrequency: 0s\nrotateCertificates: true\nruntimeRequestTimeout: 0s\nshutdownGracePeriod: 0s\nshutdownGracePeriodCriticalPods: 0s\nstaticPodPath: /etc/kubernetes/manifests\nstreamingConnectionIdleTimeout: 0s\nsyncFrequency: 0s\nvolumeStatsAggPeriod: 0s\n"}}
I0207 08:22:07.879657 46264 round_trippers.go:466] curl -v -XPOST -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s'
I0207 08:22:07.881675 46264 round_trippers.go:553] POST https://10.189.20.64:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 1 milliseconds
I0207 08:22:07.881699 46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:07.881711 46264 round_trippers.go:577] Response Headers:
I0207 08:22:07.881722 46264 round_trippers.go:580] X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:07.881734 46264 round_trippers.go:580] X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:07.881744 46264 round_trippers.go:580] Content-Length: 1657
I0207 08:22:07.881755 46264 round_trippers.go:580] Date: Tue, 07 Feb 2023 00:22:07 GMT
I0207 08:22:07.881765 46264 round_trippers.go:580] Audit-Id: 96f8fa2c-8899-4cb8-8c27-2719e5f3cc0b
I0207 08:22:07.881776 46264 round_trippers.go:580] Cache-Control: no-cache, private
I0207 08:22:07.881786 46264 round_trippers.go:580] Content-Type: application/json
I0207 08:22:07.881814 46264 request.go:1154] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"kubelet-config","namespace":"kube-system","uid":"c6277399-bbe5-4b2d-ad78-94a120465f7e","resourceVersion":"207","creationTimestamp":"2023-02-07T00:22:07Z","annotations":{"kubeadm.kubernetes.io/component-config.hash":"sha256:53821686a8ddaca8403df63803a726d2ab31b145c7da69ed7b78c45d2e223a18"},"managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"v1","time":"2023-02-07T00:22:07Z","fieldsType":"FieldsV1","fieldsV1":{"f:data":{".":{},"f:kubelet":{}},"f:metadata":{"f:annotations":{".":{},"f:kubeadm.kubernetes.io/component-config.hash":{}}}}}]},"data":{"kubelet":"apiVersion: kubelet.config.k8s.io/v1beta1\nauthentication:\n anonymous:\n enabled: false\n webhook:\n cacheTTL: 0s\n enabled: true\n x509:\n clientCAFile: /etc/kubernetes/pki/ca.crt\nauthorization:\n mode: Webhook\n webhook:\n cacheAuthorizedTTL: 0s\n cacheUnauthorizedTTL: 0s\ncgroupDriver: systemd\nclusterDNS:\n- 10.96.0.10\nclusterDomain: cluster.local\ncpuManagerReconcilePeriod: 0s\nevictionPressureTransitionPeriod: 0s\nfileCheckFrequency: 0s\nhealthzBindAddress: 127.0.0.1\nhealthzPort: 10248\nhttpCheckFrequency: 0s\nimageMinimumGCAge: 0s\nkind: KubeletConfiguration\nlogging:\n flushFrequency: 0\n options:\n json:\n infoBufferSize: \"0\"\n verbosity: 0\nmemorySwap: {}\nnodeStatusReportFrequency: 0s\nnodeStatusUpdateFrequency: 0s\nrotateCertificates: true\nruntimeRequestTimeout: 0s\nshutdownGracePeriod: 0s\nshutdownGracePeriodCriticalPods: 0s\nstaticPodPath: /etc/kubernetes/manifests\nstreamingConnectionIdleTimeout: 0s\nsyncFrequency: 0s\nvolumeStatsAggPeriod: 0s\n"}}
I0207 08:22:07.881985 46264 request.go:1154] Request Body: {"kind":"Role","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:kubelet-config","namespace":"kube-system","creationTimestamp":null},"rules":[{"verbs":["get"],"apiGroups":[""],"resources":["configmaps"],"resourceNames":["kubelet-config"]}]}
I0207 08:22:07.882061 46264 round_trippers.go:466] curl -v -XPOST -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" -H "Accept: application/json, */*" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s'
I0207 08:22:07.883908 46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 201 Created in 1 milliseconds
I0207 08:22:07.883930 46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:07.883941 46264 round_trippers.go:577] Response Headers:
I0207 08:22:07.883953 46264 round_trippers.go:580] Date: Tue, 07 Feb 2023 00:22:07 GMT
I0207 08:22:07.883964 46264 round_trippers.go:580] Audit-Id: 29f655fb-97ca-41fa-9987-59bf5f73fc0b
I0207 08:22:07.883974 46264 round_trippers.go:580] Cache-Control: no-cache, private
I0207 08:22:07.883985 46264 round_trippers.go:580] Content-Type: application/json
I0207 08:22:07.883995 46264 round_trippers.go:580] X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7bb-e70e6593b237
I0207 08:22:07.884006 46264 round_trippers.go:580] X-Kubernetes-Pf-Prioritylevel-Uid: d78a58a3-d939-453f-9c68-45c1e01ca56a
I0207 08:22:07.884017 46264 round_trippers.go:580] Content-Length: 535
I0207 08:22:07.884038 46264 request.go:1154] Response Body: {"kind":"Role","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:kubelet-config","namespace":"kube-system","uid":"eb145f80-3a93-4f2c-aa45-6824597f4757","resourceVersion":"208","creationTimestamp":"2023-02-07T00:22:07Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"rbac.authorization.k8s.io/v1","time":"2023-02-07T00:22:07Z","fieldsType":"FieldsV1","fieldsV1":{"f:rules":{}}}]},"rules":[{"verbs":["get"],"apiGroups":[""],"resources":["configmaps"],"resourceNames":["kubelet-config"]}]}
I0207 08:22:07.884161 46264 request.go:1154] Request Body: {"kind":"RoleBinding","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"kubeadm:kubelet-config","namespace":"kube-system","creationTimestamp":null},"subjects":[{"kind":"Group","name":"system:nodes"},{"kind":"Group","name":"system:bootstrappers:kubeadm:default-node-token"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"Role","name":"kubeadm:kubelet-config"}}
I0207 08:22:07.884227 46264 round_trippers.go:466] curl -v -XPOST -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.24.8 (linux/amd64) kubernetes/fdc7750" 'https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s'
I0207 08:22:07.886011 46264 round_trippers.go:553] POST https://10.189.20.64:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 201 Created in 1 milliseconds
I0207 08:22:07.886032 46264 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0207 08:22:07.886044 46264 round_trippers.go:577] Response Headers:
I0207 08:22:07.886056 46264 round_trippers.go:580] Audit-Id: 81c58bd1-2cc8-4658-bc14-a3f4b1fa2ff2
I0207 08:22:07.886067 46264 round_trippers.go:580] Cache-Control: no-cache, private
I0207 08:22:07.886077 46264 round_trippers.go:580] Content-Type: application/json
I0207 08:22:07.886088 46264 round_trippers.go:580] X-Kubernetes-Pf-Flowschema-Uid: 4525dc3f-8622-4b86-b7b