Kubernetes安装之九：配置node节点之kubelet

配置node节点需要提前在node准备工作

1.docker 环境+kubectl客户端配置+kubeadm二进制文件

2.flanneld跨主机通信

可以参考前面的文章完成安装配置

3.准备初始化环境

yum install -y conntrack ipvsadm ipset jq iptables curl sysstat libseccomp
复制代码

4.修改内核参数

cat > /etc/sysctl.d/kubernetes.conf <<EOF
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
vm.swappiness=0
vm.overcommit_memory=1
vm.panic_on_oom=0
fs.inotify.max_user_watches=89100
EOF

sysctl -p /etc/sysctl.d/kubernetes.conf
复制代码

5.开启内核模块，设置防火墙规则，创建工作目录

[root@localhost ~]# modprobe br_netfilter
[root@localhost ~]# modprobe ip_vs
[root@localhost ~]# iptables -P FORWARD ACCEPT
[root@localhost ~]# mkdir -p /var/lib/{kubelet,kube-proxy}
[root@localhost ~]# 
复制代码

6.启动相关的服务

systemctl daemon-reload
systemctl enable flanneld docker
systemctl restart flanneld docker
systemctl status flanneld docker复制代码

7.node节点创建token和kubeconfig文件

[root@localhost ~]# kubectl config set-cluster kubernetes \
>       --certificate-authority=/etc/ssl/ca.pem \
>       --embed-certs=true \
>       --server=https://192.168.1.43:8443 \
>       --kubeconfig=kubelet-bootstrap.kubeconfig
Cluster "kubernetes" set.
复制代码

[root@localhost ~]# kubectl config set-credentials kubelet-bootstrap \
>       --token=pnk1m9.fvbnpd37bp35ij4w \
>       --kubeconfig=kubelet-bootstrap.kubeconfig
User "kubelet-bootstrap" set.
复制代码

[root@localhost ~]# kubectl config set-context default \
>       --cluster=kubernetes \
>       --user=kubelet-bootstrap \
>       --kubeconfig=kubelet-bootstrap.kubeconfig
Context "default" created.
复制代码

[root@localhost ~]# kubectl config use-context default --kubeconfig=kubelet-bootstrap.kubeconfig
Switched to context "default".
复制代码

8.检查token状态

[root@localhost ~]# kubeadm token list --kubeconfig ~/.kube/config
TOKEN                     TTL       EXPIRES                     USAGES                   DESCRIPTION               EXTRA GROUPS
pnk1m9.fvbnpd37bp35ij4w   23h       2019-02-15T15:02:05+08:00   authentication,signing   kubelet-bootstrap-token   system:bootstrappers:k8s-node01
复制代码

9.编写kubelet的配置文件

cat > /opt/kubernetes/cfg/kubelet.config.json <<EOF
{
  "kind": "KubeletConfiguration",
  "apiVersion": "kubelet.config.k8s.io/v1beta1",
  "authentication": {
    "x509": {
      "clientCAFile": "/etc/ssl/ca.pem"
    },
    "webhook": {
      "enabled": true,
      "cacheTTL": "2m0s"
    },
    "anonymous": {
      "enabled": false
    }
  },
  "authorization": {
    "mode": "Webhook",
    "webhook": {
      "cacheAuthorizedTTL": "5m0s",
      "cacheUnauthorizedTTL": "30s"
    }
  },
  "address": "192.168.1.44",
  "port": 10250,
  "readOnlyPort": 0,
  "cgroupDriver": "cgroupfs",
  "hairpinMode": "promiscuous-bridge",
  "serializeImagePulls": false,
  "featureGates": {
    "RotateKubeletClientCertificate": true,
    "RotateKubeletServerCertificate": true
  },
  "clusterDomain": "cluster.local.",
  "clusterDNS": ["10.254.0.2"]
}
EOF
复制代码

• 说明：
• address：API 监听地址，不能为0.0.1，否则 kube-apiserver、heapster 等不能调用 kubelet 的 API
• readOnlyPort=0：关闭只读端口(默认 10255)，等效为未指定
• anonymous.enabled：设置为 false，不允许匿名访问 10250 端口
• x509.clientCAFile：指定签名客户端证书的 CA 证书，开启 HTTP 证书认证
• webhook.enabled=true：开启 HTTPs bearer token 认证
• 对于未通过 x509 证书和 webhook 认证的请求(kube-apiserver 或其他客户端)，将被拒绝，提示 Unauthorized
• mode=Webhook：kubelet 使用 SubjectAccessReview API 查询 kube-apiserver 某 user、group 是否具有操作资源的权限(RBAC)
• RotateKubeletClientCertificate、featureGates.RotateKubeletServerCertificate：自动 rotate 证书，证书的有效期取决于 kube-controller-manager 的 --experimental-cluster-signing-duration 参数
• 需要 root 账户运行

10.配置kuebelet服务

cat > /usr/lib/systemd/system/kubelet.service <<EOF
[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=docker.service
Requires=docker.service

[Service]
WorkingDirectory=/var/lib/kubelet
ExecStart=/usr/bin/kubelet \\
  --bootstrap-kubeconfig=/opt/kubernetes/cfg/kubelet-bootstrap.kubeconfig \\
  --cert-dir=/etc/ssl \\
  --kubeconfig=/opt/kubernetes/cfg/kubelet.kubeconfig \\
  --config=/opt/kubernetes/cfg/kubelet.config.json \\
  --hostname-override=k8s-node01 \\
  --pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest \\
  --allow-privileged=true \\
  --alsologtostderr=true \\
  --logtostderr=false \\
  --log-dir=/var/log/kubernetes \\
  --v=2
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF
复制代码

11.Bootstrap Token Auth 和授予权限

[root@localhost ~]# kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --group=system:bootstrappers
clusterrolebinding.rbac.authorization.k8s.io/kubelet-bootstrap created
复制代码

12.启动服务

systemctl daemon-reload
systemctl enable kubelet
systemctl restart kubelet
systemctl status kubelet复制代码

13.创建kube证书

cat > /etc/ssl/kube-proxy/kube-proxy-csr.json <<EOF
{
  "CN": "system:kube-proxy",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "ChengDu",
      "L": "ChengDu",
      "O": "k8s",
      "OU": "dessler"
    }
  ]
}
EOF
复制代码

cfssl gencert -ca=/etc/ssl/ca.pem \
  -ca-key=/etc/ssl/ca-key.pem \
  -config=/etc/ssl/ca-config.json \
  -profile=kubernetes  kube-proxy-csr.json | cfssljson -bare kube-proxy
复制代码

[root@host40 kube-proxy]# ls
kube-proxy.csr  kube-proxy-csr.json  kube-proxy-key.pem  kube-proxy.pem
复制代码

14.分发证书，及二进制文件

15.配置kubeconfig文件

root@localhost kube-proxy]# kubectl config set-cluster kubernetes \
>   --certificate-authority=/etc/ssl/ca.pem \
>   --embed-certs=true \
>   --server=https://192.168.1.43:8443 \
>   --kubeconfig=kube-proxy.kubeconfig
Cluster "kubernetes" set.
复制代码

[root@localhost kube-proxy]# kubectl config set-credentials kube-proxy \
>   --client-certificate=/etc/ssl/kube-proxy/kube-proxy.pem \
>   --client-key=/etc/ssl/kube-proxy/kube-proxy-key.pem \
>   --embed-certs=true \
>   --kubeconfig=kube-proxy.kubeconfig
User "kube-proxy" set.
复制代码

[root@localhost kube-proxy]# kubectl config set-context default \
>   --cluster=kubernetes \
>   --user=kube-proxy \
>   --kubeconfig=kube-proxy.kubeconfig
Context "default" created.
复制代码

[root@localhost kube-proxy]# kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig
Switched to context "default".
复制代码

16.准备配置文件

cat > /opt/kubernetes/cfg/kube-proxy.config.yaml <<EOF
apiVersion: kubeproxy.config.k8s.io/v1alpha1
bindAddress: 192.168.1.44
clientConnection:
  kubeconfig: /opt/kubernetes/cfg/kube-proxy.kubeconfig
clusterCIDR: 172.30.0.0/16
healthzBindAddress: 192.168.1.44:10256
hostnameOverride: k8s-node01
kind: KubeProxyConfiguration
metricsBindAddress: 192.168.1.44:10249
mode: "ipvs"
EOF
复制代码

17.准备kube-proxy systemctl启动文件

• 说明：
• bindAddress: 监听地址
• kubeconfig: 连接 apiserver 的 kubeconfig 文件
• clusterCIDR: 必须与 kube-controller-manager 的--cluster-cidr 选项值一致；kube-proxy 根据 --cluster-cidr 判断集群内部和外部流量，指定 --cluster-cidr 或 --masquerade-all 选项后 kube-proxy 才会对访问 Service IP 的请求做 SNAT
• hostnameOverride: 参数值必须与 kubelet 的值一致，否则 kube-proxy 启动后会找不到该 Node，从而不会创建任何 ipvs 规则
• mode: 使用 ipvs 模式

18.启动服务

systemctl daemon-reload
systemctl enable kube-proxy
systemctl restart kube-proxy
systemctl status kube-proxy
复制代码