k8s 二进制安装过程错误日志记录（持续更新）

问题1

apiserver日志

I1123 15:15:42.595711    3704 clientconn.go:948] ClientConn switching balancer to "pick_first"
I1123 15:15:42.595979    3704 client.go:360] parsed scheme: "passthrough"
I1123 15:15:42.595999    3704 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://192.168.0.13:2379  <nil> 0 <nil>}] <nil> <nil>}
I1123 15:15:42.596004    3704 clientconn.go:948] ClientConn switching balancer to "pick_first"
W1123 15:15:42.597192    3704 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://192.168.0.11  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp: address 192.168.0.11: missing port in address". Reconnecting...
I1123 15:15:42.597337    3704 balancer_conn_wrappers.go:78] pickfirstBalancer: HandleSubConnStateChange: 0xc000e384d0, {CONNECTING <nil>}
I1123 15:15:42.597361    3704 balancer_conn_wrappers.go:78] pickfirstBalancer: HandleSubConnStateChange: 0xc000e384d0, {TRANSIENT_FAILURE connection error: desc = "transport: Error while dialing dial tcp: address 192.168.0.11: missing port in address"}
I1123 15:15:42.597574    3704 balancer_conn_wrappers.go:78] pickfirstBalancer: HandleSubConnStateChange: 0xc000e386e0, {CONNECTING <nil>}
I1123 15:15:42.597609    3704 balancer_conn_wrappers.go:78] pickfirstBalancer: HandleSubConnStateChange: 0xc000e388d0, {CONNECTING <nil>}
I1123 15:15:42.616643    3704 balancer_conn_wrappers.go:78] pickfirstBalancer: HandleSubConnStateChange: 0xc000e386e0, {READY <nil>}
I1123 15:15:42.616813    3704 store.go:1376] Monitoring customresourcedefinitions.apiextensions.k8s.io count at <storage-prefix>//apiextensions.k8s.io/customresourcedefinitions
I1123 15:15:42.640755    3704 cacher.go:405] cacher (*apiextensions.CustomResourceDefinition): initialized
I1123 15:15:42.641178    3704 balancer_conn_wrappers.go:78] pickfirstBalancer: HandleSubConnStateChange: 0xc000e388d0, {READY <nil>}
I1123 15:15:42.645636    3704 instance.go:289] Using reconciler: lease
I1123 15:15:42.645999    3704 client.go:360] parsed scheme: "endpoint"
I1123 15:15:42.646017    3704 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://192.168.0.11  <nil> 0 <nil>} {https://192.168.0.12:2379  <nil> 0 <nil>} {https://192.168.0.13:2379  <nil> 0 <nil>}]
W1123 15:15:42.646549    3704 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://192.168.0.11  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp: address 192.168.0.11: missing port in address". Reconnecting...
I1123 15:15:42.648476    3704 controlbuf.go:508] transport: loopyWriter.run returning. connection error: desc = "transport is closing"
I1123 15:15:42.648607    3704 controlbuf.go:508] transport: loopyWriter.run returning. connection error: desc = "transport is closing"
I1123 15:15:42.657862    3704 cacher.go:405] cacher (*apiextensions.CustomResourceDefinition): initialized
E1123 15:15:42.659017    3704 instance.go:392] Could not construct pre-rendered responses for ServiceAccountIssuerDiscovery endpoints. Endpoints will not be enabled. Error: issuer URL must use https scheme, got: api
I1123 15:15:42.659504    3704 client.go:360] parsed scheme: "endpoint"
I1123 15:15:42.659526    3704 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://192.168.0.11  <nil> 0 <nil>} {https://192.168.0.12:2379  <nil> 0 <nil>} {https://192.168.0.13:2379  <nil> 0 <nil>}]
W1123 15:15:42.659740    3704 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://192.168.0.11  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp: address 192.168.0.11: missing port in address". Reconnecting...
I1123 15:15:42.669289    3704 store.go:1376] Monitoring podtemplates count at <storage-prefix>//podtemplates
I1123 15:15:42.669876    3704 client.go:360] parsed scheme: "endpoint"
I1123 15:15:42.669894    3704 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://192.168.0.11  <nil> 0 <nil>} {https://192.168.0.12:2379  <nil> 0 <nil>} {https://192.168.0.13:2379  <nil> 0 <nil>}]
W1123 15:15:42.670183    3704 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://192.168.0.11  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp: address 192.168.0.11: missing port in address". Reconnecting...
I1123 15:15:42.678617    3704 store.go:1376] Monitoring events count at <storage-prefix>//events

解决方案：apiserver中指定的etcd服务配置错误，检查配置修改即可

问题2

controller日志

I1123 15:27:56.981261    3775 leaderelection.go:243] attempting to acquire leader lease kube-system/kube-controller-manager...
E1123 15:27:56.982248    3775 leaderelection.go:325] error retrieving resource lock kube-system/kube-controller-manager: Get "http://https:192.168.0.11:8443/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager?timeout=10s": dial tcp: lookup https:192.168.0.11: no such host
I1123 15:27:56.982322    3775 tlsconfig.go:240] Starting DynamicServingCertificateController
E1123 15:27:59.059451    3775 leaderelection.go:325] error retrieving resource lock kube-system/kube-controller-manager: Get "http://https:192.168.0.11:8443/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager?timeout=10s": dial tcp: lookup https:192.168.0.11: no such host
E1123 15:28:03.280244    3775 leaderelection.go:325] error retrieving resource lock kube-system/kube-controller-manager: Get "http://https:192.168.0.11:8443/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager?timeout=10s": dial tcp: lookup https:192.168.0.11: no such host
E1123 15:28:07.035608    3775 leaderelection.go:325] error retrieving resource lock kube-system/kube-controller-manager: Get "http://https:192.168.0.11:8443/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager?timeout=10s": dial tcp: lookup https:192.168.0.11: no such host

解决方案：kubectl config 配置错误，检查修改

问题3

failed to run Kubelet: misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs"

解决方案：修改docker，只需在/etc/docker/daemon.json中，添加"exec-opts": [“native.cgroupdriver=systemd”]即可，本文最初的docker配置可供参考。

问题4

查看kube-proxy日志出现：

Failed to retrieve node info: nodes "192.168.122.102" is forbidden: User "system:anonymous" cannot get resource "nodes" in API group "" at the cluster scope

解决方案：这是因为kube-apiserver的RBAC

[root@k8s-master master]# kubectl create clusterrolebinding the-boss --user system:anonymous --clusterrole cluster-admin

问题5

kubelet日志

Failed to get system container stats for "/user.slice/user-0.slice/session-3.scope": failed to get cgroup stats for "/user.slice/user-0.slice/session-3.scope": failed to get container info for "/user.slice/user-0.slice/session-3.scope": unknown container "/user.slice/user-0.slice/session-3.scope"

解决方案：在kubelet中追加配置

--runtime-cgroups=/systemd/system.slice --kubelet-cgroups=/systemd/system.slice