k8s1.29.4高可用集群部署

最新推荐文章于 2024-05-06 17:21:08 发布
最新推荐文章于 2024-05-06 17:21:08 发布
阅读量1.1k 收藏 21
点赞数 20
文章标签： kubernetes 容器 云原生
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/rocyan2019/article/details/137937681
版权

一. 软件、镜像清单

1 软件清单

软件版本
kubelet1.29.4-150500.2.1.x86_64
kubernetes-cni1.3.0-150500.1.1.x86_64
kubectl1.29.4-150500.2.1.x86_64
kubeadm1.29.4-150500.2.1.x86_64
containerd.io1.6.31-3.1.el7.x86_64

2 镜像清单

IMAGETAG
docker.io/calico/cniv3.26.1
docker.io/calico/nodev3.26.1
registry.cn-hangzhou.aliyuncs.com/google_containers/corednsv1.11.1
registry.cn-hangzhou.aliyuncs.com/google_containers/etcd3.5.12-0
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserverv1.29.4
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-managerv1.29.4
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxyv1.29.4
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-schedulerv1.29.4
registry.cn-hangzhou.aliyuncs.com/google_containers/pause3.9
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgenv1.4.0
registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-serverv0.5.2
registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controllerv1.10.0

二. 主机信息

hostname主机ip系统版本
baseos192.168.3.35CentOS 7.9.2009
master1192.168.3.34CentOS 7.9.2009
master2192.168.3.36CentOS 7.9.2009
master3192.168.3.38CentOS 7.9.2009
Node1192.168.1.114CentOS 7.9.2009
Node2192.168.1.115CentOS 7.9.2009

三.部署nginx，用做k8s apiserver负载均衡

sudo yum install epel-release
sudo yum install nginx -y
yum -y install nginx-all-modules.noarch
sudo systemctl start nginx
sudo systemctl enable nginx
cat > /etc/nginx/nginx.conf << "EOF"
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
 
include /usr/share/nginx/modules/*.conf;
 
events {
    worker_connections 1024;
}
 
# 四层负载均衡，为Master apiserver提供负载均衡
stream {
 
    log_format  main  '$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent';
 
    access_log  /var/log/nginx/k8s-access.log  main;
 
    upstream k8s-apiserver {
       server 192.168.3.34:6443;
       server 192.168.3.36:6443;
       server 192.168.3.38:6443;
    }
    
    server {
       listen 6443; # 如果nginx与master节点部署在一起，这个监听端口不能是6443，否则会冲突
       proxy_pass k8s-apiserver;
    }
}
 
http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
 
    access_log  /var/log/nginx/access.log  main;
 
    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;
 
    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;
 
    server {
        listen       80 default_server;
        server_name  _;
 
        location / {
        }
    }
}
EOF
# 重启nginx
sudo systemctl restart nginx

四. 系统基础配置（所有节点）

1.关闭selinux，关闭防火墙，关闭交换分区

# 关闭防火墙
systemctl stop firewalld
systemctl disable firewalld
# 关闭selinux
setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
# 关闭swap
sed -i 's/.swap./#&/' /etc/fstab
swapoff -a
# 设置主机名
hostnamectl set-hostname master1
# 配置host解析
cat >> /etc/hosts << EOF
192.168.3.10  master1
192.168.3.11  node1
192.168.3.15  node2
EOF
# 将桥接的IPv4流量传递到iptables的链：
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
net.ipv4.tcp_tw_recycle=0
vm.swappiness=0
vm.overcommit_memory=1
vm.panic_on_oom=0
fs.inotify.max_user_instances=8192
fs.inotify.max_user_watches=1048576
fs.file-max=52706963
fs.nr_open=52706963
net.ipv6.conf.all.disable_ipv6=1
net.netfilter.nf_conntrack_max=2310720         
EOF
sysctl --system
# 添加加载的内核模块
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
# 加载内核模块
sudo modprobe overlay
sudo modprobe br_netfilter
# 设置内核参数
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF
# 应用内核参数
sudo sysctl --system
# 验证参数是否生效
lsmod | grep br_netfilter
lsmod | grep overlay
sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward
# 时间同步
yum install chrony -y
systemctl start chronyd && systemctl enable chronyd && chronyc sources
# 修改时区
ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
# 修改语言
sudo echo 'LANG="en_US.UTF-8"' >> /etc/profile;source /etc/profile
# 安装ipvs
yum -y install ipset ipvsadm
mkdir -p /etc/sysconfig/ipvsadm
cat > /etc/sysconfig/ipvsadm/ipvs.modules <<EOFmodprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack
EOF
chmod 755 /etc/sysconfig/ipvsadm/ipvs.modules && bash /etc/sysconfig/ipvsadm/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack

五. 容器运行时安装（所有节点）

# 移除docker
yum -y remove docker*
# 查看是否移除成功
rpm -qa|grep docker
# 配置yum源
sudo curl -o /etc/yum.repos.d/docker-ce.repo https://download.docker.com/linux/centos/docker-ce.repo
sudo sed -i 's+download.docker.com+mirrors.aliyun.com/docker-ce+' /etc/yum.repos.d/docker-ce.repo
yum install -y containerd.io
# 初始化配置
containerd config default | sudo tee /etc/containerd/config.toml
sudo sed -i 's/\bSystemdCgroup = false\b/SystemdCgroup = true/' /etc/containerd/config.toml
sudo sed -i 's/sandbox_image = ".\+"/sandbox_image = "registry.cn-hangzhou.aliyuncs.com\/google_containers\/pause:3.9"/' /etc/containerd/config.toml
# 启动containerd
systemctl start containerd
systemctl enable containerd

六. 安装kubeadm，kubelet，kubectl（所有节点）

# 配置yum源
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.29/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.29/rpm/repodata/repomd.xml.key
exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni
EOF
# 安装kubelet kubeadm kubectl
yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
#修改kubelet的cgroup为systemd
sudo sed -i 's/^KUBELET_EXTRA_ARGS=.*/KUBELET_EXTRA_ARGS="--cgroup-driver=systemd"/' /etc/sysconfig/kubelet
sudo systemctl enable --now kubelet

七. kubeadm初始化（master1节点）

1.因kubeadm默认生成的证书有效期只有一年，所以需要下载k8s源码修改证书有效期

# 安装golang环境
rpm --import https://mirror.go-repo.io/centos/RPM-GPG-KEY-GO-REPO
curl -s https://mirror.go-repo.io/centos/go-repo.repo | tee /etc/yum.repos.d/go-repo.repo
yum install golang -y
# 下载k8s源码并解压
wget https://github.com/kubernetes/kubernetes/archive/refs/tags/v1.29.4.tar.gz
tar -zxvf v1.29.4.tar.gz
# 修改证书有效期配置
  # 修改ca证书有效期
cd kubernetes-1.29.4/staging/src/k8s.io/client-go/util/cert/
vi cert.go
 # 将如下配置有10年改成100年，如果打算做10年有效期的证书，此处可以不用改
                  NotAfter:              now.Add(duration365d * 100).UTC(),
  # 修改其余证书有效期
cd kubernetes-1.29.4/cmd/kubeadm/app/constants/
vi constants.go
    # 将如下配置由1年改成100年，如果做10年有效期的改为10年
          CertificateValidity = time.Hour * 24 * 365 * 100
# 编译kubeadm
cd kubernetes-1.29.4/
make WHAT=cmd/kubeadm GOFLAGS=-v
# 备份kubeadm，并将编译后的kubeadm复制到/usr/bin/目录下
mv /usr/bin/kubeadm /usr/bin/kubeadm.old
cp _output/bin/kubeadm /usr/bin/
# 生成初始化文件
kubeadm config print init-defaults > kubeadm-init.yaml

2. 配置 kubeadm-init.yaml

apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 192.168.3.34 # 配置master节点ip
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///var/run/containerd/containerd.sock  # 使用的容器运行时
  imagePullPolicy: IfNotPresent
  name: master1   # 节点名称
  taints: null
---
apiServer:
  timeoutForControlPlane: 4m0s
  certSANs:  # 添加所有master节点信息
  - 192.168.3.34
  - 192.168.3.35
  - 192.168.3.36
  - 192.168.3.37
  - master1
  - master2
  - master3
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: "192.168.3.35:6443"  # master节点vip配置
controllerManager: {}
dns: {}
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers  # 镜像仓库，使用阿里镜像仓库
kind: ClusterConfiguration
kubernetesVersion: 1.29.4
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.96.0.0/12  # service网段
  podSubnet: 30.100.0.0/16  # 指定pod网段
scheduler: {}

3. 初始化master1节点

# 初始化master节点
kubeadm init --config=kubeadm-init.yaml

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of control-plane nodes by copying certificate authorities
and service account keys on each node and then running the following as root:

  kubeadm join 192.168.3.35:6443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:bc586fecb737f0e560b7a151d6efe289209a12affe5a8d5c5f980df6182de21e \
        --control-plane

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.3.35:6443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:bc586fecb737f0e560b7a151d6efe289209a12affe5a8d5c5f980df6182de21e

# 初始化成功 后根据提示执行如下命令
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl get node

八. 安装网络插件（master1节点）

# 下载calico.yaml
wget https://raw.githubusercontent.com/projectcalico/calico/v3.26.1/manifests/calico.yaml
vim calico.yaml
# 配置pod网段以及网卡
            - name: CALICO_IPV4POOL_CIDR
              value: "30.100.0.0/16"
            - name: IP_AUTODETECTION_METHOD
              value: "interface=eth0"
kubectl apply -f calico.yaml

九. master节点加入集群

#从master1节点拷贝证书
# master2节点
mkdir -p /etc/kubernetes/pki/etcd &&mkdir -p ~/.kube/
scp /etc/kubernetes/admin.conf root@192.168.3.36:/etc/kubernetes/
scp /etc/kubernetes/pki/ca.* root@192.168.3.36:/etc/kubernetes/pki/
scp /etc/kubernetes/pki/sa.* root@192.168.3.36:/etc/kubernetes/pki/
scp /etc/kubernetes/pki/front-proxy-ca.* root@192.168.3.36:/etc/kubernetes/pki/
scp /etc/kubernetes/pki/etcd/ca.* root@192.168.3.36:/etc/kubernetes/pki/etcd/
# master3节点
mkdir -p /etc/kubernetes/pki/etcd &&mkdir -p ~/.kube/
scp /etc/kubernetes/admin.conf root@192.168.3.38:/etc/kubernetes/
scp /etc/kubernetes/pki/ca.* root@192.168.3.38:/etc/kubernetes/pki/
scp /etc/kubernetes/pki/sa.* root@192.168.3.38:/etc/kubernetes/pki/
scp /etc/kubernetes/pki/front-proxy-ca.* root@192.168.3.38:/etc/kubernetes/pki/
scp /etc/kubernetes/pki/etcd/ca.* root@192.168.3.38:/etc/kubernetes/pki/etcd/

# 根据master1节点初始化生成的命令，到其他几个master节点执行
  kubeadm join 192.168.3.35:6443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:bc586fecb737f0e560b7a151d6efe289209a12affe5a8d5c5f980df6182de21e \
        --control-plane
# 初始化成功 后根据提示执行如下命令
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl get node
# 所有master节点放开service的端口限制
vim /etc/kubernetes/manifests/kube-apiserver.yaml
# 新增如下参数修改service的端口范围限制
- --service-node-port-range=1-65535
# 重启kubelet
systemctl restart kubelet

十. node节点加入集群

# 根据master1节点初始化生成的命令，到其node节点执行
kubeadm join 192.168.3.35:6443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:bc586fecb737f0e560b7a151d6efe289209a12affe5a8d5c5f980df6182de21e

十一. 检查集群状态

[root@master1 ~]# kubectl get nodes
NAME      STATUS   ROLES           AGE     VERSION
master1   Ready    control-plane   18h     v1.29.4
master2   Ready    control-plane   17h     v1.29.4
master3   Ready    control-plane   6h      v1.29.4
node1     Ready    <none>          6h21m   v1.29.4
node2     Ready    <none>          6h20m   v1.29.4

十二. 安装ingress

wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/cloud/deploy.yaml
sed -i 's+registry.k8s.io/ingress-nginx/controller:v1.10.0@sha256:42b3f0e5d0846876b1791cd3afeb5f1cbbe4259d6f35651dcc1b5c980925379c+registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.10.0+' deploy.yaml
sed -i 's+registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.0@sha256:44d1d0e9f19c63f58b380c5fddaca7cf22c7cee564adeff365225a5df5ef3334+registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.4.0+' deploy.yaml
kubectl apply -f deploy.yaml

vim ingress-nginx-deploy.yaml

apiVersion: v1
kind: Namespace
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx
  namespace: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - namespaces
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - configmaps
  - pods
  - secrets
  - endpoints
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses/status
  verbs:
  - update
- apiGroups:
  - networking.k8s.io
  resources:
  - ingressclasses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - coordination.k8s.io
  resourceNames:
  - ingress-nginx-leader
  resources:
  - leases
  verbs:
  - get
  - update
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - create
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - discovery.k8s.io
  resources:
  - endpointslices
  verbs:
  - list
  - watch
  - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx-admission
  namespace: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
  - create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  - endpoints
  - nodes
  - pods
  - secrets
  - namespaces
  verbs:
  - list
  - watch
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses/status
  verbs:
  - update
- apiGroups:
  - networking.k8s.io
  resources:
  - ingressclasses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - discovery.k8s.io
  resources:
  - endpointslices
  verbs:
  - list
  - watch
  - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx-admission
rules:
- apiGroups:
  - admissionregistration.k8s.io
  resources:
  - validatingwebhookconfigurations
  verbs:
  - get
  - update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx
subjects:
- kind: ServiceAccount
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx-admission
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx
subjects:
- kind: ServiceAccount
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx-admission
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: v1
data:
  allow-snippet-annotations: "false"
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx-controller
  namespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  externalTrafficPolicy: Local
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - appProtocol: http
    name: http
    port: 80
    protocol: TCP
    targetPort: http
  - appProtocol: https
    name: https
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx-controller-admission
  namespace: ingress-nginx
spec:
  ports:
  - appProtocol: https
    name: https-webhook
    port: 443
    targetPort: webhook
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  minReadySeconds: 0
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/component: controller
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/name: ingress-nginx
  strategy:
    rollingUpdate:
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      labels:
        app.kubernetes.io/component: controller
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.10.0
    spec:
      containers:
      - args:
        - /nginx-ingress-controller
        - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
        - --election-id=ingress-nginx-leader
        - --controller-class=k8s.io/ingress-nginx
        - --ingress-class=nginx
        - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
        - --validating-webhook=:8443
        - --validating-webhook-certificate=/usr/local/certificates/cert
        - --validating-webhook-key=/usr/local/certificates/key
        - --enable-metrics=false
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: LD_PRELOAD
          value: /usr/local/lib/libmimalloc.so
        image: registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.10.0
        imagePullPolicy: IfNotPresent
        lifecycle:
          preStop:
            exec:
              command:
              - /wait-shutdown
        livenessProbe:
          failureThreshold: 5
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: controller
        ports:
        - containerPort: 80
          name: http
          protocol: TCP
        - containerPort: 443
          name: https
          protocol: TCP
        - containerPort: 8443
          name: webhook
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources:
          requests:
            cpu: 100m
            memory: 90Mi
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            add:
            - NET_BIND_SERVICE
            drop:
            - ALL
          readOnlyRootFilesystem: false
          runAsNonRoot: true
          runAsUser: 101
          seccompProfile:
            type: RuntimeDefault
        volumeMounts:
        - mountPath: /usr/local/certificates/
          name: webhook-cert
          readOnly: true
      dnsPolicy: ClusterFirst
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: ingress-nginx
      terminationGracePeriodSeconds: 300
      volumes:
      - name: webhook-cert
        secret:
          secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx-admission-create
  namespace: ingress-nginx
spec:
  template:
    metadata:
      labels:
        app.kubernetes.io/component: admission-webhook
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.10.0
      name: ingress-nginx-admission-create
    spec:
      containers:
      - args:
        - create
        - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
        - --namespace=$(POD_NAMESPACE)
        - --secret-name=ingress-nginx-admission
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        image: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.4.0
        imagePullPolicy: IfNotPresent
        name: create
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          runAsUser: 65532
          seccompProfile:
            type: RuntimeDefault
      nodeSelector:
        kubernetes.io/os: linux
      restartPolicy: OnFailure
      serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx-admission-patch
  namespace: ingress-nginx
spec:
  template:
    metadata:
      labels:
        app.kubernetes.io/component: admission-webhook
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.10.0
      name: ingress-nginx-admission-patch
    spec:
      containers:
      - args:
        - patch
        - --webhook-name=ingress-nginx-admission
        - --namespace=$(POD_NAMESPACE)
        - --patch-mutating=false
        - --secret-name=ingress-nginx-admission
        - --patch-failure-policy=Fail
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        image: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.4.0
        imagePullPolicy: IfNotPresent
        name: patch
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          runAsUser: 65532
          seccompProfile:
            type: RuntimeDefault
      nodeSelector:
        kubernetes.io/os: linux
      restartPolicy: OnFailure
      serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: nginx
spec:
  controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:
  - v1
  clientConfig:
    service:
      name: ingress-nginx-controller-admission
      namespace: ingress-nginx
      path: /networking/v1/ingresses
  failurePolicy: Fail
  matchPolicy: Equivalent
  name: validate.nginx.ingress.kubernetes.io
  rules:
  - apiGroups:
    - networking.k8s.io
    apiVersions:
    - v1
    operations:
    - CREATE
    - UPDATE
    resources:
    - ingresses
  sideEffects: None

十三. 部署metrics

kubectl apply -f components.yaml

vim components.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    k8s-app: metrics-server
    rbac.authorization.k8s.io/aggregate-to-admin: "true"
    rbac.authorization.k8s.io/aggregate-to-edit: "true"
    rbac.authorization.k8s.io/aggregate-to-view: "true"
  name: system:aggregated-metrics-reader
rules:
- apiGroups:
  - metrics.k8s.io
  resources:
  - pods
  - nodes
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    k8s-app: metrics-server
  name: system:metrics-server
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - nodes
  - nodes/stats
  - namespaces
  - configmaps
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server-auth-reader
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: extension-apiserver-authentication-reader
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server:system:auth-delegator
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:auth-delegator
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: system:metrics-server
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:metrics-server
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
spec:
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    k8s-app: metrics-server
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
spec:
  selector:
    matchLabels:
      k8s-app: metrics-server
  strategy:
    rollingUpdate:
      maxUnavailable: 0
  template:
    metadata:
      labels:
        k8s-app: metrics-server
    spec:
      containers:
      - args:
        - --cert-dir=/tmp
        - --secure-port=4443
        - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
        - --kubelet-use-node-status-port
        - --metric-resolution=15s
        - --kubelet-insecure-tls
        image: registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-server:v0.5.2 # 修改为国内镜像地址
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /livez
            port: https
            scheme: HTTPS
          periodSeconds: 10
        name: metrics-server
        ports:
        - containerPort: 4443
          name: https
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /readyz
            port: https
            scheme: HTTPS
          initialDelaySeconds: 20
          periodSeconds: 10
        resources:
          requests:
            cpu: 100m
            memory: 200Mi
        securityContext:
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          runAsUser: 1000
        volumeMounts:
        - mountPath: /tmp
          name: tmp-dir
      nodeSelector:
        kubernetes.io/os: linux
      priorityClassName: system-cluster-critical
      serviceAccountName: metrics-server
      volumes:
      - emptyDir: {}
        name: tmp-dir
---
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
  labels:
    k8s-app: metrics-server
  name: v1beta1.metrics.k8s.io
spec:
  group: metrics.k8s.io
  groupPriorityMinimum: 100
  insecureSkipTLSVerify: true
  service:
    name: metrics-server
    namespace: kube-system
  version: v1beta1
  versionPriority: 100
roc_yan2019
关注
  • 20
    点赞
  • 踩
  • 21
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 1
    评论
k8s 1.25.4 高可用集群二进制部署文档
02-14
通过二进制部署k8s高可用集群，包括haproxy、keepalived、kube-apiserver、kube-controller-manager、kube-scheduler、kubectl、kubelet、kube-proxy组件，使用Container作为容器组件，另外搭建Calico、Core DNS组件...
【k8s】（二）kubernetes1.29.4离线部署之-镜像文件准备
Qinghub‘博客
04-19 1480
kubernetes1.29.4离线部署之-镜像文件提前下载
k8s安装部署详细教程
wuxiong1028的博客
02-01 6160
修改私服在jenkins服务器的/etc/docker/daemon.json# 重新加载将镜像推送到harbor。
基于centos7的k8s最新版v1.29.2安装教程
xingdiango的博客
03-14 3244
Kubernetes 是一个可移植、可扩展的开源平台，用于管理容器化的工作负载和服务，可促进声明式配置和自动化。Kubernetes 拥有一个庞大且快速增长的生态，其服务、支持和工具的使用范围相当广泛。这个名字源于希腊语，意为“舵手”或“飞行员”。k8s 这个缩写是因为 k 和 s 之间有八个字符的关系。Google 在 2014 年开源了 Kubernetes 项目。Kubernetes 建立在的基础上， 结合了社区中最优秀的想法和实践。
k8s1.29.0 集群部署
lucas5的博客
03-26 522
k8s1.29.0 集群部署，kubeadm 3master+3slave keepalived+haproxy flannel0.24-host-gw calico3.27-bgp-rr istio1.21.1 efk系统 elasticsearch8.13.1+logstash8.13.1+kibana8.13.1+filebeat8.13.1+kafka-cluster3.7.0+elastalert2部署 + reindex索引迁移
kubeadm快速部署Kubernetes 1.29.0版本集群
qq_46268601的博客
12-27 3541
新版本Kubernetes 1.29.0搭建
Kubernetes1.29 部署
微光
02-22 2449
Ubuntu22.04 部署 Kubernetes 集群
Docker K8S高可用集群部署手册
最新发布
08-04
Docker K8S高可用集群部署手册： CentOS系统默认的内核版本是3.10，对于k8s-v1.24及以后版本来说，在生产环境上能部署，能运行，但是在使用k8s期间会出现很多问题，即不稳定因数。为了能在生产环境中稳定运行，对于...
二进制部署k8s高可用集群（二进制-V1.20）.docx
06-29
二进制部署k8s高可用集群（二进制-V1.20） 基于提供的文件信息，本文将对二进制部署k8s高可用集群进行详细的知识点总结。 1. 生产环境部署 K8s 集群的两种方式 在生产环境中，部署 K8s 集群有两种方式：一种是...
kubeadm初始化高可用k8s1.20.4集群—超详细的文档带软件包
06-12
kubeadm初始化高可用k8s1.20.4集群详细资料，超详细，带部署文档和相关软件包，保证能顺利部署完成
k8s1.20.4-高可用集群部署-新增项目-kubernetes安装包和详细文档笔记整理
05-29
在本资源中，我们聚焦于"Kubernetes 1.20.4 高可用集群的部署，这是一项关键的IT技术，尤其对于那些寻求优化云计算基础设施的企业和开发者来说。Kubernetes，简称K8s，是Google开源的一个容器编排系统，能够自动化...
【k8s】（四）kubernetes1.29.4离线部署之-组件安装
Qinghub‘博客
04-20 600
kubernetes1.29.4离线部署之-组件安装
【k8s】Kubernetes 1.29.4离线安装部署（总）
Qinghub‘博客
04-19 4788
Kubernetes 1.29.4 离线安装部署；脚本化快速部署；容器运行时采用Containerd；逐步支持多平台部署。
k8s的安装部署，详细过程展示（保姆级安装教程）
m0_58833554的博客
11-10 4058
k8s的详细安装部署过程
【k8s】搭建小巧完备的Kubernetes环境（minikube）
L李钟意的博客
03-20 2470
Kubernetes 就是一个生产级别的容器编排平台和集群管理系统，不仅能够创建、调度容器，还能够监控、管理服务器，它凝聚了 Google 等大公司和开源社区的集体智慧，从而让中小型公司也可以具备轻松运维海量计算节点——也就是“云计算”的能力。
安装k8s v1.29记录
qq_35573061的博客
05-06 835
#####node节点安装。
Kubernetes介绍
区块链领域填坑的java程序猿
07-31 1169
Kubernetes简单介绍　　Kubernetes是什么？ Kubernetes是一个基于Docker容器的开源编制系统，它能在跨多个主机上管理Docker应用，并提供应用程序部署 维护和扩展的基本机制。　　它透明地为用户提供原生态系统，如“需要5个 WildFly服务器和1个 MySQL服务器运行”. Kubernetes具有自我修复机制，如重新启动 重新启动定时计划 复制容器以确保恢复状态，用
云原生Kubernetes: K8S 1.29版本 部署ingress-nginx
cronaldo91的博客
04-21 1714
原因是node1节点的cni容器出现了异常无法为pod分配ip导致的卡在ContainerCreating的状态。ingress-nginx-controller部署到node1节点的IP为10.244.166.140。ingress-nginx-controller部署到node2节点的IP为10.244.104.13。ingress-nginx-controller部署到node1节点的IP为10.244.104.13。④在自己的仓库中可以看到上传的镜像，默认上传到公共仓库中。
评论 1
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

roc_yan2019

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值