【Kubernetes硬核部署方式kubernetes-the-hard-way】06-部署kubernetes工作节点
六、部署kubernetes工作节点
这部分我们部署kubernetes的工作节点。实例中我们有两个工作节点,一个是独立的工作节点,一个是跟master在一起的节点。
在每个节点上我们会部署kubelet、kube-proxy、container runtime、cni、nginx-proxy
下面的操作需要在每一个工作节点执行
1. Container Runtime - Containerd
1.1 软件包下载
# 设定containerd的版本号
$ VERSION=1.7.15
# 下载压缩包
$ wget https://github.com/containerd/containerd/releases/download/v${VERSION}/cri-containerd-cni-${VERSION}-linux-amd64.tar.gz
#可以在node-1下载之后进行分发至各个工作节点
$ scp cri-containerd-cni-${VERSION}-linux-amd64.tar.gz node-2:~
$ scp cri-containerd-cni-${VERSION}-linux-amd64.tar.gz node-3:~
1.2 整理压缩文件
下载后的文件是一个tar.gz,是一个allinone的包,包括了runc、circtl、ctr、containerd等容器运行时以及cni相关的文件,解压缩到一个独立的目录中
# 设定containerd的版本号
$ VERSION=1.7.15
# 创建一个containerd文件夹并将文件移动至这个文件夹
$ mkdir containerd
$ mv cri-containerd-cni-${VERSION}-linux-amd64.tar.gz containerd/
$ cd containerd
# 解压缩
$ tar -xvf cri-containerd-cni-${VERSION}-linux-amd64.tar.gz
# 复制需要的文件
$ cp etc/crictl.yaml /etc/
$ cp etc/systemd/system/containerd.service /etc/systemd/system/
$ cp -r usr /
1.3 containerd配置文件
$ mkdir -p /etc/containerd
# 默认配置生成配置文件
$ containerd config default > /etc/containerd/config.toml
# 定制化配置(可选)
$ vi /etc/containerd/config.toml
1.4 启动containerd
$ systemctl enable containerd
$ systemctl restart containerd
# 检查状态
$ systemctl status containerd
2. 配置kubelet
准备kubelet配置
#返回结点
$ cd
$ mkdir -p /etc/kubernetes/ssl/
$ mv ${HOSTNAME}-key.pem ${HOSTNAME}.pem ca.pem ca-key.pem /etc/kubernetes/ssl/
$ mv ${HOSTNAME}.kubeconfig /etc/kubernetes/kubeconfig
# 当前工作结点的IP
$ IP=192.168.31.71
# 写入kubelet配置文件
$ cat <<EOF > /etc/kubernetes/kubelet-config.yaml
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
anonymous:
enabled: false
webhook:
enabled: true
x509:
clientCAFile: "/etc/kubernetes/ssl/ca.pem"
authorization:
mode: Webhook
clusterDomain: "cluster.local"
clusterDNS:
- "169.254.25.10"
podCIDR: "10.200.0.0/16"
address: ${IP}
readOnlyPort: 0
staticPodPath: /etc/kubernetes/manifests
healthzPort: 10248
healthzBindAddress: 127.0.0.1
kubeletCgroups: /systemd/system.slice
resolvConf: "/etc/resolv.conf"
runtimeRequestTimeout: "15m"
kubeReserved:
cpu: 200m
memory: 512M
tlsCertFile: "/etc/kubernetes/ssl/${HOSTNAME}.pem"
tlsPrivateKeyFile: "/etc/kubernetes/ssl/${HOSTNAME}-key.pem"
EOF
配置kubelet服务
$ cat <<EOF > /etc/systemd/system/kubelet.service
[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/kubernetes/kubernetes
After=containerd.service
Requires=containerd.service
[Service]
ExecStart=/usr/local/bin/kubelet \\
--config=/etc/kubernetes/kubelet-config.yaml \\
--container-runtime=remote \\
--container-runtime-endpoint=unix:///var/run/containerd/containerd.sock \\
--image-pull-progress-deadline=2m \\
--kubeconfig=/etc/kubernetes/kubeconfig \\
--network-plugin=cni \\
--node-ip=${IP} \\
--register-node=true \\
--v=2
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
EOF
3. 配置nginx-proxy
nginx-proxy是一个用于worker节点访问apiserver的一个代理,是apiserver一个优雅的高可用方案,它使用kubelet的staticpod方式启动,让每个节点都可以均衡的访问到每个apiserver服务,优雅的替代了通过虚拟ip访问apiserver的方式。
Tips: nginx-proxy 只需要在没有 apiserver 的节点部署,这里我们配置到node-3结点
3.1 nginx配置文件
$ mkdir -p /etc/nginx
# master ip列表
$ MASTER_IPS=(192.168.31.70 192.168.31.71)
# 执行前请先copy一份,并修改好upstream的 'server' 部分配置
$ cat <<EOF > /etc/nginx/nginx.conf
error_log stderr notice;
worker_processes 2;
worker_rlimit_nofile 130048;
worker_shutdown_timeout 10s;
events {
multi_accept on;
use epoll;
worker_connections 16384;
}
stream {
upstream kube_apiserver {
least_conn;
server ${MASTER_IPS[0]}:6443;
server ${MASTER_IPS[1]}:6443;
}
server {
listen 127.0.0.1:6443;
proxy_pass kube_apiserver;
proxy_timeout 10m;
proxy_connect_timeout 1s;
}
}
http {
aio threads;
aio_write on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 5m;
keepalive_requests 100;
reset_timedout_connection on;
server_tokens off;
autoindex off;
server {
listen 8081;
location /healthz {
access_log off;
return 200;
}
location /stub_status {
stub_status on;
access_log off;
}
}
}
EOF
3.2 nginx manifest
$ mkdir -p /etc/kubernetes/manifests/
$ cat <<EOF > /etc/kubernetes/manifests/nginx-proxy.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx-proxy
namespace: kube-system
labels:
addonmanager.kubernetes.io/mode: Reconcile
k8s-app: kube-nginx
spec:
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet
nodeSelector:
kubernetes.io/os: linux
priorityClassName: system-node-critical
containers:
- name: nginx-proxy
image: docker.io/library/nginx:1.19
imagePullPolicy: IfNotPresent
resources:
requests:
cpu: 25m
memory: 32M
securityContext:
privileged: true
livenessProbe:
httpGet:
path: /healthz
port: 8081
readinessProbe:
httpGet:
path: /healthz
port: 8081
volumeMounts:
- mountPath: /etc/nginx
name: etc-nginx
readOnly: true
volumes:
- name: etc-nginx
hostPath:
path: /etc/nginx
EOF
4. 配置kube-proxy
4.1 配置文件
$ mv kube-proxy.kubeconfig /etc/kubernetes/
# 创建 kube-proxy-config.yaml
$ cat <<EOF > /etc/kubernetes/kube-proxy-config.yaml
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
bindAddress: 0.0.0.0
clientConnection:
kubeconfig: "/etc/kubernetes/kube-proxy.kubeconfig"
clusterCIDR: "10.200.0.0/16"
mode: ipvs
EOF
4.2 kube-proxy 服务文件
$ cat <<EOF > /etc/systemd/system/kube-proxy.service
[Unit]
Description=Kubernetes Kube Proxy
Documentation=https://github.com/kubernetes/kubernetes
[Service]
ExecStart=/usr/local/bin/kube-proxy \\
--config=/etc/kubernetes/kube-proxy-config.yaml
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
EOF
5. 启动服务
$ systemctl daemon-reload
$ systemctl enable kubelet kube-proxy
$ systemctl restart kubelet kube-proxy
$ journalctl -f -u kubelet
$ journalctl -f -u kube-proxy
5.1找不到这个文件夹"/etc/kubernetes/manifests"问题![外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传](https://img-home.csdnimg.cn/images/20230724024159.png?origin_url=C%3A%5CUsers%5Cchina%5CDesktop%5Ckubernetes-the-hard-way%5Cdocs%5C%E9%97%AE%E9%A2%983.png&pos_id=img-gg7alO3N-1713450332753
#创建manifests文件夹
mkdir -p /etc/kubernetes/manifests
6. 手动下载镜像(服务器无法访问外网情况)
在每个工作节点下载pause镜像
$ crictl pull registry.cn-hangzhou.aliyuncs.com/kubernetes-kubespray/pause:3.2
$ ctr -n k8s.io i tag registry.cn-hangzhou.aliyuncs.com/kubernetes-kubespray/pause:3.2 k8s.gcr.io/pause:3.2