部署一套高可用的k8s集群

已于 2024-05-05 15:40:31 修改
阅读量678 收藏 14
点赞数 9
分类专栏： 运维 计算机基础 文章标签： linux 服务器 运维 docker etcd 容器 kubernetes
于 2024-04-18 11:32:36 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/qq_64519337/article/details/137914293
版权

前言：

这套部署是典型的 Kubernetes 集群架构，包含了 Nginx + Keepalived、Etcd、Docker 和 Harbor 这些组件，各自扮演着不同的角色，共同构建了一个稳定、高可用的容器化环境。以下是对这些组件的介绍和作用：

  1. Nginx + Keepalived:

    • Nginx: 作为一个高性能的反向代理服务器和负载均衡器，Nginx 负责接收来自外部用户的请求，并将这些请求转发给后端的 Kubernetes API Server 或者其他服务。

    • Keepalived: 用于实现高可用性和故障转移。通过 Keepalived 的配置，可以将多个 Nginx 实例组成一个虚拟 IP（VIP）集群，确保即使其中一个 Nginx 节点出现故障，服务仍然可用。

  2. Etcd:

    • Etcd 是一个分布式键值存储系统，主要用于存储 Kubernetes 集群的各种配置信息、状态信息和资源对象的数据。它保证了集群中各个节点之间的数据一致性，并支持高可用部署，确保集群的稳定性和可靠性。

  3. Docker:

    • Docker 是一个开源的容器化平台，用于打包、分发和运行应用程序。在 Kubernetes 集群中，Docker 负责管理和运行各个容器，提供了轻量级、快速部署的容器化解决方案。

  4. Harbor:

    • Harbor 是一个企业级的容器镜像仓库，用于存储和管理 Docker 镜像。在 Kubernetes 集群中，Harbor 可以作为集中式的镜像仓库，提供镜像的版本管理、权限控制、安全扫描等功能，简化了容器镜像的管理和使用。

这套部署架构的优点包括：

  • 高可用性: 使用了 Nginx + Keepalived 和 Etcd 实现了集群的高可用性和故障转移，确保服务的连续性和稳定性。

  • 容器化管理: 使用 Docker 将应用程序打包成容器，实现了轻量级、快速部署的容器化管理。

  • 镜像管理: 使用 Harbor 作为容器镜像仓库，实现了容器镜像的集中管理、安全扫描和权限控制。

综合来看，这套部署结合了容器化技术和高可用架构，为 Kubernetes 集群提供了稳定、可靠的运行环境，适用于生产环境的部署和管理。

硬件环境

4台虚机 : 2c2g,20g

硬件环境

软件环境

一、基础环境配置

1.系统安装已经IP配置：

下载CentOS Linux release 7.9.2009 (Core)镜像

安装完成之后，进行IP配置：

vi /etc/sysconfig/network-scripts/ifcfg-ens33 
​
##修改
BOOTPROTO="static"    #这表示网络接口使用静态 IP 地址配置，而不是动态获取 IP 地址（比如通过 DHCP）
ONBOOT="yes"          #这表示在系统启动时自动启用这个网络接口
​
#添加
IPADDR="99.99.10.10"   #静态 IP 地址
GATEWAY="99.99.10.2"    #网关地址，用于指示该网络接口的数据包应该通过哪个网关发送到其他网络
NETMASK="255.255.255.0"    #子网掩码，用于确定该网络接口所在网络的范围
DNS1=99.99.10.2         #DNS 服务器的 IP 地址，用于系统解析域名
​
:wq                        

重启network：

systemctl restart netwok

ping baidu.com测试网络：

[root@k8s-master1 ~]# ping 99.99.10.14
PING 99.99.10.14 (99.99.10.14) 56(84) bytes of data.
64 bytes from 99.99.10.14: icmp_seq=1 ttl=64 time=0.104 ms
64 bytes from 99.99.10.14: icmp_seq=2 ttl=64 time=0.080 ms
64 bytes from 99.99.10.14: icmp_seq=3 ttl=64 time=0.048 ms
64 bytes from 99.99.10.14: icmp_seq=4 ttl=64 time=0.038 ms
64 bytes from 99.99.10.14: icmp_seq=5 ttl=64 time=0.041 ms
​
[root@k8s-master1 data]# ping baidu.com
PING baidu.com (39.156.66.10) 56(84) bytes of data.
64 bytes from 39.156.66.10 (39.156.66.10): icmp_seq=1 ttl=128 time=11.8 ms
64 bytes from 39.156.66.10 (39.156.66.10): icmp_seq=2 ttl=128 time=16.0 ms
64 bytes from 39.156.66.10 (39.156.66.10): icmp_seq=3 ttl=128 time=12.7 ms
64 bytes from 39.156.66.10 (39.156.66.10): icmp_seq=4 ttl=128 time=12.1 ms

2.集群名称规划：

在每台的/etc/hosts上加入
cat >> /etc/hosts << EOF
99.99.10.10 k8s-master1
99.99.10.11 k8s-master2
99.99.10.12 k8s-node1
99.99.10.13 k8s-harbor.com
EOF
​

3.基础环境配置

#关闭防火墙
systemctl stop firewalld && systemctl disable  firewalld
#关闭NetworkManager，NetworkManager会和network冲突
systemctl stop NetworkManager && systemctl disable  NetworkManager
​
#暂时关闭 SELinux，并将其配置为禁用状态
setenforce 0
sed -i s/SELINUX=enforcing/SELINUX=disabled/ /etc/selinux/config
​
#关闭swap 分区，并将 /etc/fstab 文件中关于 swap 分区的条目注释，以防止系统在下次启动时重新启用 swap 分区
swapoff -a
sed -ri 's/.*swap.*/#&/' /etc/fstab
​
​
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system  
​
​
yum install ntpdate -y
#使用 ntpdate 工具来同步系统时间
ntpdate time.windows.com

4.设置免密

ssh-keygen          #连续回车即可
​
ssh-copy-id -i ~/.ssh/id_rsa.pub root@99.99.10.11
ssh-copy-id -i ~/.ssh/id_rsa.pub root@99.99.10.12
ssh-copy-id -i ~/.ssh/id_rsa.pub root@99.99.10.13

二、部署Nginx+Keepalived负载均衡器

1.安装nginx和keepalived

yum install epel-release -y
yum install nginx keepalived -y
#这样安装nginx缺少stream模块，需要手动安装
yum install -y nginx-mod-stream

2.修改nginx配置文件

cat > /etc/nginx/nginx.conf << "EOF"
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
​
include /usr/share/nginx/modules/*.conf;
​
events {
    worker_connections 1024;
}
​
# 四层负载均衡，为两台Master apiserver组件提供负载均衡
# 这个strem是为nginx4层负载均衡的一个模块，不使用的https 7层的负载均衡；
stream {
​
    log_format  main  '$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent';
​
    access_log  /var/log/nginx/k8s-access.log  main;
​
    upstream k8s-apiserver {
       server 99.99.10.10:6443;   # Master1 APISERVER IP:PORT
       server 99.99.10.11:6443;   # Master2 APISERVER IP:PORT
    }
​
    server {
       listen 16443;  # 由于nginx与master节点复用，这个监听端口不能是6443，否则会冲突
       proxy_pass k8s-apiserver;
    }
}
​
http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
​
    access_log  /var/log/nginx/access.log  main;
​
    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;
​
    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;
}
EOF

3.修改keepalived配置文件

cat > /etc/keepalived/keepalived.conf << EOF
global_defs { 
   notification_email { 
     acassen@firewall.loc 
     failover@firewall.loc 
     sysadmin@firewall.loc 
   } 
   notification_email_from Alexandre.Cassen@firewall.loc  
   smtp_server 127.0.0.1 
   smtp_connect_timeout 30 
   router_id NGINX_MASTER
} 
​
vrrp_script check_nginx {
    script "/etc/keepalived/check_nginx.sh"
}
​
vrrp_instance VI_1 { 
    state MASTER 
    interface ens33  # 修改为实际网卡名
    virtual_router_id 51 # VRRP 路由 ID实例，每个实例是唯一的 
    priority 100    # 优先级，备服务器设置 90 
    advert_int 1    # 指定VRRP 心跳包通告间隔时间，默认1秒 
    authentication { 
        auth_type PASS      
        auth_pass 1111 
    }  
    # 虚拟IP
    virtual_ipaddress { 
        99.99.10.14/16
    } 
    track_script {
        check_nginx
    } 
}
EOF
cat > /etc/keepalived/keepalived.conf << EOF
global_defs { 
   notification_email { 
     acassen@firewall.loc 
     failover@firewall.loc 
     sysadmin@firewall.loc 
   } 
   notification_email_from Alexandre.Cassen@firewall.loc  
   smtp_server 127.0.0.1 
   smtp_connect_timeout 30 
   router_id NGINX_BACKUP
} 
​
vrrp_script check_nginx {
    script "/etc/keepalived/check_nginx.sh"
}
​
vrrp_instance VI_1 { 
    state BACKUP 
    interface ens33
    virtual_router_id 51 # VRRP 路由 ID实例，每个实例是唯一的 
    priority 90 #backup这里为90
    advert_int 1
    authentication { 
        auth_type PASS      
        auth_pass 1111 
    }  
    virtual_ipaddress { 
        99.99.10.14/16 #VIP的IP
    } 
    track_script {
        check_nginx
    } 
}
EOF

4.启动并设置开机自启

systemctl daemon-reload
systemctl start nginx
systemctl start keepalived
systemctl enable nginx
systemctl enable keepalived

如果出现：nginx: [error] invalid PID number "" in "/run/nginx.pid" 再确认安装了nginx的stream模块后

[root@k8s-master1 data]# cat /run/nginx.pid
[root@k8s-master1 data]# nginx
[root@k8s-master1 data]# cat /run/nginx.pid
2787
如果出现报错：
[root@k8s-master2 data]# systemctl start nginx
Job for nginx.service failed because the control process exited with error code. See "systemctl status nginx.service" and "journalctl -xe" for details.
[root@k8s-master2 data]# systemctl status nginx
● nginx.service - The nginx HTTP and reverse proxy server
   Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Mon 2024-04-15 14:09:47 CST; 7s ago
  Process: 17914 ExecStartPre=/usr/sbin/nginx -t (code=exited, status=1/FAILURE)
  Process: 17912 ExecStartPre=/usr/bin/rm -f /run/nginx.pid (code=exited, status=0/SUCCESS)
​
Apr 15 14:09:46 k8s-master2 systemd[1]: Starting The nginx HTTP and reverse proxy server...
Apr 15 14:09:47 k8s-master2 nginx[17914]: nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
Apr 15 14:09:47 k8s-master2 nginx[17914]: nginx: [emerg] bind() to 0.0.0.0:16443 failed (13: Permission denied)
Apr 15 14:09:47 k8s-master2 nginx[17914]: nginx: configuration file /etc/nginx/nginx.conf test failed
Apr 15 14:09:47 k8s-master2 systemd[1]: nginx.service: control process exited, code=exited status=1
Apr 15 14:09:47 k8s-master2 systemd[1]: Failed to start The nginx HTTP and reverse proxy server.
Apr 15 14:09:47 k8s-master2 systemd[1]: Unit nginx.service entered failed state.
Apr 15 14:09:47 k8s-master2 systemd[1]: nginx.service failed.

  1. 检查SELinux的状态。如果SELinux处于强制模式，可能会阻止Nginx在非标准端口上监听。你可以使用以下命令来检查SELinux的状态：

getenforce

如果显示的是Enforcing，你可能需要调整SELinux的策略允许Nginx在该端口上绑定，或者你可以临时将SELinux设置为Permissive（宽容模式）进行测试：

sudo setenforce 0

  1. 检查是否有防火墙规则可能在阻止Nginx绑定指定的端口。如果你的系统使用的是firewalld，可以用下列命令来允许端口16443：

sudo firewall-cmd --permanent --zone=public --add-port=16443/tcp

  1. 创建nginx.pid

vim /run/nginx.pid

  1. 重新启动nginx

[root@k8s-master2 data]# sudo systemctl stop nginx
[root@k8s-master2 data]# sudo systemctl start nginx
[root@k8s-master2 data]# systemctl status nginx
● nginx.service - The nginx HTTP and reverse proxy server
   Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2024-04-15 14:13:20 CST; 7s ago
  Process: 17977 ExecStart=/usr/sbin/nginx (code=exited, status=0/SUCCESS)
  Process: 17973 ExecStartPre=/usr/sbin/nginx -t (code=exited, status=0/SUCCESS)
  Process: 17972 ExecStartPre=/usr/bin/rm -f /run/nginx.pid (code=exited, status=0/SUCCESS)
 Main PID: 17979 (nginx)
   CGroup: /system.slice/nginx.service
           ├─17979 nginx: master process /usr/sbin/nginx
           ├─17980 nginx: worker process
           └─17981 nginx: worker process
​
Apr 15 14:13:20 k8s-master2 systemd[1]: Starting The nginx HTTP and reverse proxy server...
Apr 15 14:13:20 k8s-master2 nginx[17973]: nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
Apr 15 14:13:20 k8s-master2 nginx[17973]: nginx: configuration file /etc/nginx/nginx.conf test is successful
Apr 15 14:13:20 k8s-master2 systemd[1]: Started The nginx HTTP and reverse proxy server.
​

5.查看keepalived状态

[root@k8s-master1 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:36:37:bd brd ff:ff:ff:ff:ff:ff
    inet 99.99.10.10/24 brd 99.99.10.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 99.99.10.14/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::e4b9:35be:db90:55b/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::8df6:8f68:1ae:e989/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::72b5:c53b:5302:401d/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:4c:bc:c4:7f brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
​

这里网卡ens33成功绑定了99.99.10.14虚拟IP

6.测试nginx+keepalived的高可用性

关闭主节点的nginx，观察备份节点状态

[root@k8s-master2 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:e9:c5:15 brd ff:ff:ff:ff:ff:ff
    inet 99.99.10.11/24 brd 99.99.10.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::8df6:8f68:1ae:e989/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
    inet6 fe80::e4b9:35be:db90:55b/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:52:87:eb:11 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
​
[root@k8s-master1 nginx]# systemctl status nginx
● nginx.service - The nginx HTTP and reverse proxy server
   Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; vendor preset: disabled)
   Active: inactive (dead) since Mon 2024-04-15 14:30:57 CST; 11s ago
  Process: 2637 ExecStart=/usr/sbin/nginx (code=exited, status=0/SUCCESS)
  Process: 2634 ExecStartPre=/usr/sbin/nginx -t (code=exited, status=0/SUCCESS)
  Process: 2632 ExecStartPre=/usr/bin/rm -f /run/nginx.pid (code=exited, status=0/SUCCESS)
 Main PID: 2639 (code=exited, status=0/SUCCESS)
​
Apr 15 14:02:13 k8s-master1 systemd[1]: Starting The nginx HTTP and reverse proxy server...
Apr 15 14:02:13 k8s-master1 nginx[2634]: nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
Apr 15 14:02:13 k8s-master1 nginx[2634]: nginx: configuration file /etc/nginx/nginx.conf test is successful
Apr 15 14:02:13 k8s-master1 systemd[1]: Started The nginx HTTP and reverse proxy server.
Apr 15 14:30:57 k8s-master1 systemd[1]: Stopping The nginx HTTP and reverse proxy server...
Apr 15 14:30:57 k8s-master1 systemd[1]: Stopped The nginx HTTP and reverse proxy server.
​
​
[root@k8s-master1 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:36:37:bd brd ff:ff:ff:ff:ff:ff
    inet 99.99.10.10/24 brd 99.99.10.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 99.99.10.14/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::e4b9:35be:db90:55b/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::8df6:8f68:1ae:e989/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::72b5:c53b:5302:401d/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:4c:bc:c4:7f brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever

注意：keepalived在切换时会丢一个包，可以尝试ping 99.99.10.14虚拟IP，然后再尝试切换就可以看到切换过程中会丢一个包。

7.keepalived介绍：

Keepalived 是一个用于实现高可用性和负载均衡的开源软件。它主要用于在多个服务器之间实现故障转移，确保在其中一个服务器故障时服务仍然可用。下面是 Keepalived 的一些主要特点和用途：

  1. 故障检测：Keepalived 可以监控服务器的健康状态，如检测服务是否可用、服务器负载等。一旦发现故障，它可以自动切换到备用服务器，实现故障转移。

  2. 虚拟 IP（VIP）：Keepalived 可以为一组服务器提供一个虚拟 IP 地址（VIP），对外提供服务。如果主服务器出现故障，Keepalived 会将 VIP 转移到备用服务器，确保服务的连续性。

  3. 负载均衡：除了故障转移外，Keepalived 还可以实现负载均衡，将请求均匀分配到多个服务器上，提高系统的整体性能和可扩展性。

  4. 配置简单：Keepalived 的配置相对简单，通常使用简洁的文本配置文件即可实现基本的功能。它支持多种配置选项，可以根据具体需求进行灵活配置。

  5. 支持多种协议：Keepalived 支持多种协议，如VRRP（Virtual Router Redundancy Protocol）、SMTP（Simple Mail Transfer Protocol）、HTTP（HyperText Transfer Protocol）等，可以用于不同类型的应用场景。

总体而言，Keepalived 是一个强大的工具，可以帮助系统管理员轻松实现高可用性和负载均衡，提高系统的稳定性和可靠性。

三、部署Etcd集群

1.etcd在k8s集群的作用：

Etcd 在 Kubernetes 集群中的作用包括：

  1. 配置存储: Etcd 存储了 Kubernetes 集群的各种配置信息，如节点信息、网络配置、存储配置、安全配置等。这些配置对于集群的正常运行和管理都至关重要。

  2. 状态监控: Etcd 还负责监控集群中各个组件的状态，包括 API Server、Controller Manager、Scheduler 等组件的运行状态，以及节点的健康状态等。通过监控这些状态，可以及时发现并处理集群中的问题。

  3. 资源存储: Kubernetes 中的各种资源对象，如 Pod、Service、Replication Controller、ConfigMap 等都存储在 Etcd 中。这些对象的创建、更新、删除等操作都会反映在 Etcd 中，确保集群中各个组件的一致性和可靠性。

  4. 分布式一致性: Etcd 使用 Raft 算法来实现分布式一致性，确保集群中的数据在多个节点之间保持一致。这样可以防止单点故障，并提高集群的可用性和稳定性。

总体来说，Etcd 是 Kubernetes 集群的核心组件之一，承担着存储和管理集群配置、状态和资源数据的重要任务，对于集群的正常运行和高可用性至关重要。

2.安装cfssl证书生成工具

wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
​
chmod +x cfssl*
for x in cfssl*; do mv $x ${x%*_linux-amd64};  done
mv cfssl* /usr/bin

3.生成证书

mkdir -p ~/etcd_tls
cd ~/etcd_tls

自签CA:

cat > ca-config.json << EOF
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "www": {
         "expiry": "87600h",
         "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ]
      }
    }
  }
}
EOF
​
cat > ca-csr.json << EOF
{
    "CN": "etcd CA",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Beijing",
            "ST": "Beijing"
        }
    ]
}
EOF

生成证书：

cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
​
​
[root@k8s-master1 etcd_tls]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
2024/04/15 14:52:37 [INFO] generating a new CA key and certificate from CSR
2024/04/15 14:52:37 [INFO] generate received request
2024/04/15 14:52:37 [INFO] received CSR
2024/04/15 14:52:37 [INFO] generating key: rsa-2048
2024/04/15 14:52:37 [INFO] encoded CSR
2024/04/15 14:52:37 [INFO] signed certificate with serial number 106940532372652583388643549123311178715600450059
​

可以看到生成了ca.pem和ca-key.pem

[root@k8s-master1 etcd_tls]# ll
total 20
-rw-r--r--. 1 root root  287 Apr 15 14:52 ca-config.json
-rw-r--r--. 1 root root  956 Apr 15 14:52 ca.csr
-rw-r--r--. 1 root root  209 Apr 15 14:52 ca-csr.json
-rw-------. 1 root root 1675 Apr 15 14:52 ca-key.pem
-rw-r--r--. 1 root root 1265 Apr 15 14:52 ca.pem
​

使用自签CA签发Etcd HTTPS证书

cat > server-csr.json << EOF
{
    "CN": "etcd",
    "hosts": [
    "99.99.10.10",
    "99.99.10.11",
    "99.99.10.12",
    "99.99.10.14",
    "99.99.10.2"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "BeiJing",
            "ST": "BeiJing"
        }
    ]
}
EOF

生成证书：

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server


[root@k8s-master1 etcd_tls]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server
2024/04/15 14:54:04 [INFO] generate received request
2024/04/15 14:54:04 [INFO] received CSR
2024/04/15 14:54:04 [INFO] generating key: rsa-2048
2024/04/15 14:54:04 [INFO] encoded CSR
2024/04/15 14:54:04 [INFO] signed certificate with serial number 84218250271244398651603471024412481684503411417
2024/04/15 14:54:04 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").

可以看到生成server.pem和server-key.pem文件

[root@k8s-master1 etcd_tls]# ll
total 36
-rw-r--r--. 1 root root  287 Apr 15 14:52 ca-config.json
-rw-r--r--. 1 root root  956 Apr 15 14:52 ca.csr
-rw-r--r--. 1 root root  209 Apr 15 14:52 ca-csr.json
-rw-------. 1 root root 1675 Apr 15 14:52 ca-key.pem
-rw-r--r--. 1 root root 1265 Apr 15 14:52 ca.pem
-rw-r--r--. 1 root root 1021 Apr 15 14:54 server.csr
-rw-r--r--. 1 root root  319 Apr 15 14:53 server-csr.json
-rw-------. 1 root root 1675 Apr 15 14:54 server-key.pem
-rw-r--r--. 1 root root 1346 Apr 15 14:54 server.pem

4.安装Etcd

下载地址：https://link.zhihu.com/?target=https%3A//github.com/etcd-io/etcd/releases/download/v3.4.9/etcd-v3.4.9-linux-amd64.tar.gz 
​
#解压
tar -zxvf etcd-v3.4.9-linux-amd64.tar.gz
​
mkdir /opt/etcd/{bin,cfg,ssl} -p
mv etcd-v3.4.9-linux-amd64/{etcd,etcdctl} /opt/etcd/bin/

5.修改etcd配置文件

cat > /opt/etcd/cfg/etcd.conf << EOF
#[Member]
ETCD_NAME="etcd-1"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://99.99.10.10:2380" #2380是 集群通信的端口；
ETCD_LISTEN_CLIENT_URLS="https://99.99.10.10:2379" #2379是指它的数据端口，其他客户端要访问etcd数据库的读写都走的是这个端口；
​
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://99.99.10.10:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://99.99.10.10:2379"
ETCD_INITIAL_CLUSTER="etcd-1=https://99.99.10.10:2380,etcd-2=https://99.99.10.11:2380,etcd-3=https://99.99.10.12:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" #一种简单的认证机制，网络里可能配置了多套k8s集群，防止误同步；
ETCD_INITIAL_CLUSTER_STATE="new"
EOF

6.systemd管理etcd

cat > /usr/lib/systemd/system/etcd.service << EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
​
[Service]
Type=notify
EnvironmentFile=/opt/etcd/cfg/etcd.conf
ExecStart=/opt/etcd/bin/etcd \
--cert-file=/opt/etcd/ssl/server.pem \
--key-file=/opt/etcd/ssl/server-key.pem \
--trusted-ca-file=/opt/etcd/ssl/ca.pem \
--peer-cert-file=/opt/etcd/ssl/server.pem \
--peer-key-file=/opt/etcd/ssl/server-key.pem \
--peer-trusted-ca-file=/opt/etcd/ssl/ca.pem \
--logger=zap
Restart=on-failure
LimitNOFILE=65536
​
[Install]
WantedBy=multi-user.target
EOF

7.给etcd上传证书

cp ~/etcd_tls/ca*pem ~/etcd_tls/server*pem /opt/etcd/ssl/

8.将配置好的etcd复制到其它两个节点

scp -r /opt/etcd/ root@99.99.10.11:/opt/
scp -r /opt/etcd/ root@99.99.10.12:/opt/
​
scp /usr/lib/systemd/system/etcd.service root@99.99.10.11:/usr/lib/systemd/system/
scp /usr/lib/systemd/system/etcd.service root@99.99.10.12:/usr/lib/systemd/system/

在其它连个节点修改/opt/etcd/cfg/etcd.conf

[root@k8s-master2 data]# cat /opt/etcd/cfg/etcd.conf
#[Member]
ETCD_NAME="etcd-2"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://99.99.10.11:2380" #2380是 集群通信的端口；
ETCD_LISTEN_CLIENT_URLS="https://99.99.10.11:2379" #2379是指它的数据端口，其他客户端要访问etcd数据库的读写都走的是这个端口；
​
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://99.99.10.11:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://99.99.10.11:2379"
ETCD_INITIAL_CLUSTER="etcd-1=https://99.99.10.10:2380,etcd-2=https://99.99.10.11:2380,etcd-3=https://99.99.10.12:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" #一种简单的认证机制，网络里可能配置了多套k8s集群，防止误同步；
ETCD_INITIAL_CLUSTER_STATE="new"
​
[root@k8s-master2 data]# cat /opt/etcd/cfg/etcd.conf
#[Member]
ETCD_NAME="etcd-3"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://99.99.10.12:2380" #2380是 集群通信的端口；
ETCD_LISTEN_CLIENT_URLS="https://99.99.10.12:2379" #2379是指它的数据端口，其他客户端要访问etcd数据库的读写都走的是这个端口；
​
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://99.99.10.12:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://99.99.10.12:2379"
ETCD_INITIAL_CLUSTER="etcd-1=https://99.99.10.10:2380,etcd-2=https://99.99.10.11:2380,etcd-3=https://99.99.10.12:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" #一种简单的认证机制，网络里可能配置了多套k8s集群，防止误同步；
ETCD_INITIAL_CLUSTER_STATE="new"
​

9.启动etcd

systemctl daemon-reload
systemctl start etcd
systemctl enable etcd

如果启动etcd的时候发现一直在进行主节点选举，可以查看端口是否被防火墙拦截了

[root@k8s-master1 ssl]# systemctl status etcd
● etcd.service - Etcd Server
   Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled; vendor preset: disabled)
   Active: activating (start) since Mon 2024-04-15 15:30:56 CST; 22s ago
 Main PID: 3439 (etcd)
   CGroup: /system.slice/etcd.service
           └─3439 /opt/etcd/bin/etcd --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --trusted-ca-file=/opt/etcd/ssl/ca.pem --peer-cert-file=/opt/etcd/s...
​
Apr 15 15:31:17 k8s-master1 etcd[3439]: {"level":"info","ts":"2024-04-15T15:31:17.609+0800","caller":"raft/raft.go:713","msg":"bd58f91ef2ee2335 became candidate at term 260"}
Apr 15 15:31:17 k8s-master1 etcd[3439]: {"level":"info","ts":"2024-04-15T15:31:17.609+0800","caller":"raft/raft.go:824","msg":"bd58f91ef2ee2335 received MsgVoteResp from...t term 260"}
Apr 15 15:31:17 k8s-master1 etcd[3439]: {"level":"info","ts":"2024-04-15T15:31:17.609+0800","caller":"raft/raft.go:811","msg":"bd58f91ef2ee2335 [logterm: 1, index: 3] se...t term 260"}
Apr 15 15:31:17 k8s-master1 etcd[3439]: {"level":"info","ts":"2024-04-15T15:31:17.609+0800","caller":"raft/raft.go:811","msg":"bd58f91ef2ee2335 [logterm: 1, index: 3] se...t term 260"}
Apr 15 15:31:18 k8s-master1 etcd[3439]: {"level":"warn","ts":"2024-04-15T15:31:18.016+0800","caller":"etcdserver/server.go:2065","msg":"failed to publish local member to cluster thr...
Apr 15 15:31:18 k8s-master1 etcd[3439]: {"level":"info","ts":"2024-04-15T15:31:18.609+0800","caller":"raft/raft.go:923","msg":"bd58f91ef2ee2335 is starting a new election at term 260"}
Apr 15 15:31:18 k8s-master1 etcd[3439]: {"level":"info","ts":"2024-04-15T15:31:18.609+0800","caller":"raft/raft.go:713","msg":"bd58f91ef2ee2335 became candidate at term 261"}
Apr 15 15:31:18 k8s-master1 etcd[3439]: {"level":"info","ts":"2024-04-15T15:31:18.609+0800","caller":"raft/raft.go:824","msg":"bd58f91ef2ee2335 received MsgVoteResp from...t term 261"}
Apr 15 15:31:18 k8s-master1 etcd[3439]: {"level":"info","ts":"2024-04-15T15:31:18.609+0800","caller":"raft/raft.go:811","msg":"bd58f91ef2ee2335 [logterm: 1, index: 3] se...t term 261"}
Apr 15 15:31:18 k8s-master1 etcd[3439]: {"level":"info","ts":"2024-04-15T15:31:18.609+0800","caller":"raft/raft.go:811","msg":"bd58f91ef2ee2335 [logterm: 1, index: 3] se...t term 261"}
Hint: Some lines were ellipsized, use -l to show in full.

解决方法：

[root@k8s-master2 data]# firewall-cmd --zone=public --add-port=2379/tcp --permanent
success
[root@k8s-master2 data]# firewall-cmd --zone=public --add-port=2380/tcp --permanent
success
[root@k8s-master2 data]# firewall-cmd --reload
success
[root@k8s-master2 data]# systemctl status etcd
● etcd.service - Etcd Server
   Loaded: loaded (/usr/lib/systemd/system/etcd.service; disabled; vendor preset: disabled)
   Active: active (running) since Mon 2024-04-15 15:34:54 CST; 10s ago
 Main PID: 18206 (etcd)
   CGroup: /system.slice/etcd.service
           └─18206 /opt/etcd/bin/etcd --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --trusted-ca-file=/opt/etcd/ssl/ca.pem --peer-cert-file=/opt/etcd/...
​

10.查看集群状态

ETCDCTL_API=3 /opt/etcd/bin/etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem --endpoints="https://99.99.10.10:2379,https://99.99.10.11:2379,https://99.99.10.12:2379" endpoint health --write-out=table
​
​
[root@k8s-master1 etcd]# ETCDCTL_API=3 /opt/etcd/bin/etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem --endpoints="https://99.99.10.10:2379,https://99.99.10.11:2379,https://99.99.10.12:2379" endpoint health --write-out=table
+--------------------------+--------+-------------+-------+
|         ENDPOINT         | HEALTH |    TOOK     | ERROR |
+--------------------------+--------+-------------+-------+
| https://99.99.10.11:2379 |   true | 11.274216ms |       |
| https://99.99.10.10:2379 |   true | 10.090251ms |       |
| https://99.99.10.12:2379 |   true | 15.350268ms |       |
+--------------------------+--------+-------------+-------+
​
[root@k8s-master2 keepalived]# ETCDCTL_API=3 /opt/etcd/bin/etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem --endpoints="https://99.99.10.10:2379,https://99.99.10.11:2379,https://99.99.10.12:2379" endpoint health --write-out=table
+--------------------------+--------+-------------+-------+
|         ENDPOINT         | HEALTH |    TOOK     | ERROR |
+--------------------------+--------+-------------+-------+
| https://99.99.10.12:2379 |   true | 13.877137ms |       |
| https://99.99.10.11:2379 |   true | 13.310408ms |       |
| https://99.99.10.10:2379 |   true |  13.64291ms |       |
+--------------------------+--------+-------------+-------+
​
[root@k8s-node1 ssl]# ETCDCTL_API=3 /opt/etcd/bin/etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem --endpoints="https://99.99.10.10:2379,https://99.99.10.11:2379,https://99.99.10.12:2379" endpoint health --write-out=table
+--------------------------+--------+-------------+-------+
|         ENDPOINT         | HEALTH |    TOOK     | ERROR |
+--------------------------+--------+-------------+-------+
| https://99.99.10.12:2379 |   true | 12.405176ms |       |
| https://99.99.10.10:2379 |   true | 10.765985ms |       |
| https://99.99.10.11:2379 |   true | 12.413707ms |       |
+--------------------------+--------+-------------+-------+

四、安装docker和harbor

1.安装docker-ce

yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

yum install -y docker-ce-20.10.7 docker-ce-cli-20.10.7 containerd.io

#配置加速器文件
mkdir -p /etc/docker
cat >> /etc/docker/daemon.json <<-EOF
{
  "registry-mirrors":["https://dockerhub.azk8s.cn",
                        "http://hub-mirror.c.163.com",
                        "http://qtid6917.mirror.aliyuncs.com",
                        "http://74f21445.m.daocloud.io",
                        "https://registry.docker-cn.com",
                        "http://hub-mirror.c.163.com",
                        "https://docker.mirrors.ustc.edu.cn"
   ],
   "insecure-registries": ["k8s-harbor.com"],
   "exec-opts": ["native.cgroupdriver=systemd"]
}
EOF
vim /etc/sysctl.conf

net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-iptables = 1

sysctl -p

systemctl daemon-reload
systemctl restart docker

2.安装cri-docker

下载cri-docker
mkdir /data/softs && cd /data/softs wget https://github.com/Mirantis/cri-dockerd/releases/download/v0.3.2/cri-dockerd-0.3.2.amd64.tgz
​
解压软件 
tar xf cri-dockerd-0.3.2.amd64.tgz 
mv cri-dockerd/cri-dockerd /usr/local/bin/
​
检查效果 
cri-dockerd --version
定制配置文件
cat > /etc/systemd/system/cri-dockerd.service<<-EOF
[Unit]
Description=CRI Interface for Docker Application Container Engine
Documentation=https://docs.mirantis.com
After=network-online.target firewalld.service docker.service
Wants=network-online.target
[Service]
Type=notify
ExecStart=/usr/local/bin/cri-dockerd --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.9
 --network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin --container-runtime-endpoint=unix:///var/run/cri-dockerd.sock --cri-dockerd-root-directory=/var/lib/dockershim --docker-endpoint=unix:///var/run/docker.sock --cri-dockerd-root-directory=/var/lib/docker
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
KillMode=process
[Install]
WantedBy=multi-user.target
EOF
定制配置
cat > /etc/systemd/system/cri-dockerd.socket <<-EOF
[Unit]
Description=CRI Docker Socket for the API
PartOf=cri-docker.service
​
[Socket]
ListenStream=/var/run/cri-dockerd.sock
SocketMode=0660
SocketUser=root
SocketGroup=docker
​
[Install]
WantedBy=sockets.target
EOF
设置服务开机自启动
systemctl daemon-reload
systemctl enable cri-dockerd.service
systemctl restart cri-dockerd.service

3.安装harbor

安装docker-compose
yum install -y docker-compose


下载软件
mkdir /data/{softs,server} -p && cd /data/softs
wget https://github.com/goharbor/harbor/releases/download/v2.5.0/harbor-offline-installer-v2.5.0.tgz

解压软件
tar -zxvf harbor-offline-installer-v2.5.0.tgz -C  /data/server/
cd /data/server/harbor/

加载镜像
docker load < harbor.v2.5.0.tar.gz
docker images

备份配置
cp harbor.yml.tmpl harbor.yml
修改配置
[root@kubernetes-register /data/server/harbor]# vim harbor.yml.tmpl
    # 修改主机名
    hostname: k8s-harbor.com
    http:
      port: 80
    #https:  注释ssl相关的部分
      #  port: 443
      #  certificate: /your/certificate/path
      #  private_key: /your/private/key/path
    # 修改harbor的登录密码
    harbor_admin_password: 123456
    # 设定harbor的数据存储目录
    data_volume: /data/server/harbor/data
配置harbor
./prepare

启动harbor
./install.sh

检查效果
docker-compose ps

定制启动文件

定制服务启动文件 /etc/systemd/system/harbor.service
[Unit]
Description=Harbor
After=docker.service systemd-networkd.service systemd-resolved.service
Requires=docker.service
Documentation=http://github.com/vmware/harbor

[Service]
Type=simple
Restart=on-failure
RestartSec=5
#需要注意harbor的安装位置
ExecStart=/usr/bin/docker-compose --file /data/server/harbor/docker-compose.yml up
ExecStop=/usr/bin/docker-compose --file /data/server/harbor/docker-compose.yml down

[Install]
WantedBy=multi-user.target
加载服务配置文件
systemctl daemon-reload
启动服务
systemctl start harbor
检查状态
systemctl status harbor
设置开机自启动
systemctl enable harbor

4.访问harbor

浏览器访问域名，用户名: admin, 密码：123456
创建用户michelxct
创建michaelxct用户专用的项目仓库，名称为 michaelxct，权限为公开的
在所有节点上登录仓库
# docker login k8s-harbor.com -u michaelxct
Password:   # 输入登录密码
​
下载镜像
docker pull busybox
​
定制镜像标签
docker tag busybox kubernetes-register.sswang.com/sswang/busybox:v0.1
​
推送镜像
docker push kubernetes-register.sswang.com/sswang/busybox:v0.1

五、k8s集群初始化

1安装kubeadm+kubelet+kunectl

添加阿里yum源

cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
yum install -y kubelet-1.20.7 kubeadm-1.20.7 kubectl-1.20.7
systemctl enable kubelet

2.初始化k8s-master1

cat > kubeadm-config.yaml << EOF
​
apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: 9037x2.tcaqnpaqkra9vsbw
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 99.99.10.10
  bindPort: 6443
nodeRegistration:
  criSocket: /var/run/dockershim.sock
  name: k8s-master1
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
---
apiServer:
  certSANs:  # 包含所有Master/LB/VIP IP，一个都不能少！为了方便后期扩容可以多写几个预留的IP。
  - k8s-master1
  - k8s-master2
  - 99.99.10.10
  - 99.99.10.11
  - 99.99.10.14
  - 99.99.10.2
  - 127.0.0.1
  extraArgs:
    authorization-mode: Node,RBAC
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: 99.99.10.14:16443 # 负载均衡虚拟IP（VIP）和端口
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  external:  # 使用外部etcd
    endpoints:
    - https://99.99.10.10:2379 # etcd集群3个节点
    - https://99.99.10.11:2379
    - https://99.99.10.12:2379
    caFile: /opt/etcd/ssl/ca.pem # 连接etcd所需证书
    certFile: /opt/etcd/ssl/server.pem
    keyFile: /opt/etcd/ssl/server-key.pem
imageRepository: registry.aliyuncs.com/google_containers # 由于默认拉取镜像地址k8s.gcr.io国内无法访问，这里指定阿里云镜像仓库地址
kind: ClusterConfiguration
kubernetesVersion: v1.20.7 # K8s版本，与上面安装的一致
networking:
  dnsDomain: cluster.local
  podSubnet: 10.244.0.0/16  # Pod网络，与下面部署的CNI网络组件yaml中保持一致
  serviceSubnet: 10.96.0.0/12  # 集群内部虚拟网络，Pod统一访问入口
scheduler: {}
​
​
EOF

初始化k8s-master1前检查nginx和keepalived,确保keepalived正常，不然会出现找不到k8s-master1的情况

systemctl restart network
systemctl restart keepalived
systemctl restart nginx
​
systemctl status network
systemctl status keepalived
systemctl status nginx
​
[root@k8s-master1 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:36:37:bd brd ff:ff:ff:ff:ff:ff
    inet 99.99.10.10/24 brd 99.99.10.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 99.99.10.14/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::e4b9:35be:db90:55b/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::8df6:8f68:1ae:e989/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::72b5:c53b:5302:401d/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:4c:bc:c4:7f brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
​
​
[root@k8s-master1 ~]# kubeadm init --config kubeadm-config.yaml
[init] Using Kubernetes version: v1.20.7
[preflight] Running pre-flight checks
    [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.7. Latest validated version: 19.03
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [k8s-master1 k8s-master2 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 99.99.10.10 99.99.10.14 99.99.10.11 99.99.10.2 127.0.0.1]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] External etcd mode: Skipping etcd/ca certificate authority generation
[certs] External etcd mode: Skipping etcd/server certificate generation
[certs] External etcd mode: Skipping etcd/peer certificate generation
[certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation
[certs] External etcd mode: Skipping apiserver-etcd-client certificate generation
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[endpoint] WARNING: port specified in controlPlaneEndpoint overrides bindPort in the controlplane address
[kubeconfig] Writing "admin.conf" kubeconfig file
[endpoint] WARNING: port specified in controlPlaneEndpoint overrides bindPort in the controlplane address
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[endpoint] WARNING: port specified in controlPlaneEndpoint overrides bindPort in the controlplane address
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[endpoint] WARNING: port specified in controlPlaneEndpoint overrides bindPort in the controlplane address
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 11.661086 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.20" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node k8s-master1 as control-plane by adding the labels "node-role.kubernetes.io/master=''" and "node-role.kubernetes.io/control-plane='' (deprecated)"
[mark-control-plane] Marking the node k8s-master1 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: 9037x2.tcaqnpaqkra9vsbw
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[endpoint] WARNING: port specified in controlPlaneEndpoint overrides bindPort in the controlplane address
[addons] Applied essential addon: kube-proxy
​
Your Kubernetes control-plane has initialized successfully!
​
To start using your cluster, you need to run the following as a regular user:
​
  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config
​
Alternatively, if you are the root user, you can run:
​
  export KUBECONFIG=/etc/kubernetes/admin.conf
​
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/
​
You can now join any number of control-plane nodes by copying certificate authorities
and service account keys on each node and then running the following as root:
​
  kubeadm join 99.99.10.14:16443 --token 9037x2.tcaqnpaqkra9vsbw \
    --discovery-token-ca-cert-hash sha256:aaf2dbf6d24b07f9cacd72c971beb9880a10e2bb30ca07a2ae7e15acf45f3944 \
    --control-plane 
​
Then you can join any number of worker nodes by running the following on each as root:
​
kubeadm join 99.99.10.14:16443 --token 9037x2.tcaqnpaqkra9vsbw \
    --discovery-token-ca-cert-hash sha256:aaf2dbf6d24b07f9cacd72c971beb9880a10e2bb30ca07a2ae7e15acf45f3944

拷贝kubectl使用的连接k8s认证文件到默认路径：

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
​
[root@k8s-master1 ~]#kubectl get node
NAME          STATUS     ROLES                  AGE     VERSION
k8s-master1   NotReady   control-plane,master   7m34s   v1.20.0
[root@k8s-master1 ~]#

3.初始化k8s-master2

[root@k8s-master2 .kube]# kubeadm join 99.99.10.14:16443 --token 9037x2.tcaqnpaqkra9vsbw     --discovery-token-ca-cert-hash sha256:aaf2dbf6d24b07f9cacd72c971beb9880a10e2bb30ca07a2ae7e15acf45f3944     --control-plane 
[preflight] Running pre-flight checks
    [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 26.0.1. Latest validated version: 19.03
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
error execution phase preflight: 
One or more conditions for hosting a new control plane instance is not satisfied.
​
failure loading certificate for CA: couldn't load the certificate file /etc/kubernetes/pki/ca.crt: open /etc/kubernetes/pki/ca.crt: no such file or directory
​
Please ensure that:
* The cluster has a stable controlPlaneEndpoint address.
* The certificates that must be shared among control plane instances are provided.
​
​
To see the stack trace of this error execute with --v=5 or higher
​

可以看到缺少证书：

将k8s-master1的证书复制到k8s-master2:

scp -r /etc/kubernetes/pki/ 99.99.10.11:/etc/kubernetes/

重新尝试加入：

[root@k8s-master2 .kube]# kubeadm join 99.99.10.14:16443 --token 9037x2.tcaqnpaqkra9vsbw     --discovery-token-ca-cert-hash sha256:aaf2dbf6d24b07f9cacd72c971beb9880a10e2bb30ca07a2ae7e15acf45f3944     --control-plane 
[preflight] Running pre-flight checks
    [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 26.0.1. Latest validated version: 19.03
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[preflight] Running pre-flight checks before initializing the new control plane instance
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Using the existing "front-proxy-client" certificate and key
[certs] Using the existing "apiserver" certificate and key
[certs] Using the existing "apiserver-kubelet-client" certificate and key
[certs] Valid certificates and keys now exist in "/etc/kubernetes/pki"
[certs] Using the existing "sa" key
[kubeconfig] Generating kubeconfig files
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[endpoint] WARNING: port specified in controlPlaneEndpoint overrides bindPort in the controlplane address
[kubeconfig] Writing "admin.conf" kubeconfig file
[endpoint] WARNING: port specified in controlPlaneEndpoint overrides bindPort in the controlplane address
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[endpoint] WARNING: port specified in controlPlaneEndpoint overrides bindPort in the controlplane address
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[check-etcd] Skipping etcd check in external mode
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
[control-plane-join] using external etcd - no local stacked instance added
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[mark-control-plane] Marking the node k8s-master2 as control-plane by adding the labels "node-role.kubernetes.io/master=''" and "node-role.kubernetes.io/control-plane='' (deprecated)"
[mark-control-plane] Marking the node k8s-master2 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
​
This node has joined the cluster and a new control plane instance was created:
​
* Certificate signing request was sent to apiserver and approval was received.
* The Kubelet was informed of the new secure connection details.
* Control plane (master) label and taint were applied to the new node.
* The Kubernetes control plane instances scaled up.
​
​
To start administering your cluster from this node, you need to run the following as a regular user:
​
    mkdir -p $HOME/.kube
    sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    sudo chown $(id -u):$(id -g) $HOME/.kube/config
​
Run 'kubectl get nodes' to see this node join the cluster.
​

拷贝kubectl使用的连接k8s认证文件到默认路径：

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
​
[root@k8s-master2 ~]#kubectl get node
NAME          STATUS     ROLES                  AGE   VERSION
k8s-master1   NotReady   control-plane,master   14m   v1.20.0
k8s-master2   NotReady   control-plane,master   30s   v1.20.0
[root@k8s-master2 ~]#

注意：由于网络插件还没有部署，还没有准备就绪 NotReady。

出现的问题：
[root@k8s-master2 .kube]# kubeadm join 99.99.10.14:16443 --token 9037x2.tcaqnpaqkra9vsbw     --discovery-token-ca-cert-hash sha256:aaf2dbf6d24b07f9cacd72c971beb9880a10e2bb30ca07a2ae7e15acf45f3944     --control-plane 
[preflight] Running pre-flight checks
    [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 26.0.1. Latest validated version: 19.03
error execution phase preflight: couldn't validate the identity of the API Server: Get "https://99.99.10.14:16443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s": dial tcp 99.99.10.14:16443: connect: connection refused
To see the stack trace of this error execute with --v=5 or higher
​
systectl restart network
systectl restart keepalived
ip a

若遇到token过期的，执行下面的指令重新生成token

[root@k8s-master1 data]# kubeadm token create --print-join-command
kubeadm join 99.99.10.14:16443 --token mj1v1g.g0im25mdfhs88ovc     --discovery-token-ca-cert-hash sha256:c8adf5ec97e9b68190ab6de03f5a7bf1e5766504d06591099aa8943a7d185613

4.测试负载均衡

[root@k8s-master2 softs]# curl -k https://99.99.10.14:16443/version
{
  "major": "1",
  "minor": "20",
  "gitVersion": "v1.20.7",
  "gitCommit": "132a687512d7fb058d0f5890f07d4121b3f0a2e2",
  "gitTreeState": "clean",
  "buildDate": "2021-05-12T12:32:49Z",
  "goVersion": "go1.15.12",
  "compiler": "gc",
  "platform": "linux/amd64"
}

可以正确获取到K8s版本信息，说明负载均衡器搭建正常。该请求数据流程：curl -> vip(nginx) -> apiserver

5.初始化k8s-node

[root@k8s-node1 opt]# kubeadm join 99.99.10.14:16443 --token 9037x2.tcaqnpaqkra9vsbw \
>     --discovery-token-ca-cert-hash sha256:aaf2dbf6d24b07f9cacd72c971beb9880a10e2bb30ca07a2ae7e15acf45f3944
[preflight] Running pre-flight checks
    [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.7. Latest validated version: 19.03
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
​
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
​
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
​

六、部署网络组件

1.部署calico

Calico是一个纯三层的数据中心网络方案，是目前Kubernetes主流的网络方案。

kubectl apply -f calico.yaml
kubectl get pod -A
​
[root@k8s-master1 data]# kubectl get pod -A
NAMESPACE              NAME                                         READY   STATUS              RESTARTS   AGE
kube-system            calico-kube-controllers-97769f7c7-crtkp      0/1     ContainerCreating   0          9m52s
kube-system            calico-node-bshw6                            0/1     Running             4          9m53s
kube-system            calico-node-dbpb2                            1/1     Running             0          9m53s
kube-system            calico-node-ss4kl                            0/1     CrashLoopBackOff    6          9m53s
kube-system            coredns-7f89b7bc75-kbs6h                     0/1     ContainerCreating   0          17h
kube-system            coredns-7f89b7bc75-kv5sz                     0/1     ContainerCreating   0          17h
kube-system            kube-apiserver-k8s-master1                   1/1     Running             0          85m
kube-system            kube-apiserver-k8s-master2                   1/1     Running             0          84m
kube-system            kube-controller-manager-k8s-master1          1/1     Running             0          85m
kube-system            kube-controller-manager-k8s-master2          1/1     Running             0          84m
kube-system            kube-proxy-4j7p7                             1/1     Running             0          85m
kube-system            kube-proxy-89z8r                             1/1     Running             0          84m
kube-system            kube-proxy-fxpkq                             1/1     Running             0          84m
kube-system            kube-scheduler-k8s-master1                   1/1     Running             0          85m
kube-system            kube-scheduler-k8s-master2                   1/1     Running             0          84m
kubernetes-dashboard   dashboard-metrics-scraper-7b59f7d4df-jvzlc   0/1     ContainerCreating   0          2m40s
kubernetes-dashboard   kubernetes-dashboard-74d688b6bc-n4r5n        0/1     ContainerCreating   0          2m40s
​

2.部署 Dashboard

Dashboard是官方提供的一个UI，可用于基本管理K8s资源。

kubectl apply -f kubernetes-dashboard.yaml
​
[root@k8s-master1 data]# kubectl apply -f kubernetes-dashboard.yaml 
namespace/kubernetes-dashboard created
serviceaccount/kubernetes-dashboard created
service/kubernetes-dashboard created
secret/kubernetes-dashboard-certs created
secret/kubernetes-dashboard-csrf created
secret/kubernetes-dashboard-key-holder created
configmap/kubernetes-dashboard-settings created
role.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
deployment.apps/kubernetes-dashboard created
service/dashboard-metrics-scraper created
deployment.apps/dashboard-metrics-scraper created
[root@k8s-master1 data]# kubectl create serviceaccount dashboard-admin -n kube-system
serviceaccount/dashboard-admin created
[root@k8s-master1 data]# kubectl create clusterrolebinding dashboard-admin --clusterrole=cluster-admin --serviceaccount=kube-system:dashboard-admin
clusterrolebinding.rbac.authorization.k8s.io/dashboard-admin created
[root@k8s-master1 data]# kubectl describe secrets -n kube-system $(kubectl -n kube-system get secret | awk '/dashboard-admin/{print $1}')
Name:         dashboard-admin-token-t6wd6
Namespace:    kube-system
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: dashboard-admin
              kubernetes.io/service-account.uid: 95be708e-d111-47e3-b697-7ef4a1d3954e
​
Type:  kubernetes.io/service-account-token
​
Data
====
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IlFNUDY3SGxOZEY3eDNKcUZaOWRGSTV1c2ZhTmdzeGRQMS10c0ZSTlpUV3MifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tdDZ3ZDYiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiOTViZTcwOGUtZDExMS00N2UzLWI2OTctN2VmNGExZDM5NTRlIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.bKoEfyd1H8_92DRasWFog45G8zCm8mXu8honFYf27F7gtMeaCFTTlrs9-3mKCcOj1Nr8-rUyV9MKzHAflHuGE0AR_Ng8z5YbFGGCiQnCc9eVsB57HsVMHp-h0bWGB0oLR8O0Ztaf5SPD6efiu-2pPLLF547GSDu_mpInJD1HLSsZRXVCiU-ZV6Fvy93aDFlWlISX1YDl14Lgp6RkhO9YPjbaUwHwlts-lgAqhdyvjWL76DaRCEzTNscGTSYZczp3qLJK2I-CGWJsTCan7R-ykiMyRJqaNVCV78OhsY1CLGQ3gCedsHEy7heS_ZhVPMFkumZAFO0kgXbw_UdncW-U8w
ca.crt:     1066 bytes
namespace:  11 bytes
​

浏览器登录node:30001

MichaelxcT
关注
专栏目录
二进制安装k8s集群+nginx负载均衡+keepalived高可用，保姆级教学
weixin_73141464的博客
03-10 2116
二进制安装k8s集群，保姆级教学
二进制部署一套高可用K8s集群-v1.24+（一）
weixin_42424397的博客
08-26 975
带你一步一步通过K8s原生二进制方式部署一套完整的高可用kubernetes集群
K8s集群简述
Winter的博客
06-27 6219
K8s集群的高可用性和容错性是通过多个Master节点和Worker节点来实现的。在集群中，Master节点负责管理整个集群的关键组件，Worker节点负责运行应用程序的Pod实例。通过多个节点的协同工作，K8s可以保证应用程序的高可用性和容错性，从而确保整个集群的稳定性和可靠性。云原生应用是指在云环境下设计、构建和部署的应用程序，具有高度的可伸缩性、弹性和可靠性。它们通常使用容器化技术进行打包和部署，并使用自动化的工具进行管理和升级。云原生应用的设计原则包括微服务架构、故障隔离、自动化部署和水平扩展等。
k8S 高可用部署，超详细
最新发布
Linux运维老纪的博客
08-29 2165
Kubernetes 高可用是保证 Master 节点中的 API Server 服务的高可用。API Server 提 供了 Kubernetes 各类资源对象增删改查的唯一访问入口，是整个 Kubernetes 系统的数据 总线和数据中心。采用负载均衡（Load Balance）连接两个 Master 节点可以提供稳定容器 云业务。本章主要学习 Kubernetes 高可用的部署方法。
搭建k8s高可用集群（1.27.2版本）
05-30 1533
修改配置文件中第13行cidr，与6.5配置文件 kubeadm-config.yaml 中的 podSubnet 想同。注意：如果不加最后的--cri-socket unix://var/run/cri-dockerd.sock，会报错。其中"exec-opts": ["native.cgroupdriver=systemd"]为cgroup方式。185.199.108.133 是我查出来的IP，需要换成自己查出来的IP。输入 yes ，输入每台服务器密码。检查etcd是否正常。
开篇：使用kubeadm搭建高可用k8s集群
cheney的专栏
02-02 1426
基于haproxy + keepalived，kubeadm安装高可用kubernets 1.24 集群
搭建k8s高可用集群（Keepalived 和 HAproxy）
这里是火火火的世界
03-20 3858
spec:hosts:中每一行代表一个机器，要把所有机器列出来，name：实例的主机名（就是第一步准备工作时设置的hostname）,address和internalAddress是每个机器的IP地址，后面的user和password是能远程登录机器的用户名和密码。所有的master节点和worker工作节点配置docker镜像加速器（这里额外添加了docker的生产环境核心配置cgroup）使用如下命令在所有的master节点和worker工作节点安装docker并配置docker镜像加速器。
部署一套完整的K8s高可用集群（kubeadm-V1.21）
05-09
部署一套完整的 K8s 高可用集群（kubeadm-V1.21） Kubernetes 作为容器集群系统，通过健康检查+重启策略实现了 Pod 故障自我修复能力，通过调度算法实现将 Pod 分布式部署，并保持预期副本数，根据 Node 失效状态...
部署一套完整的K8s集群课件v1.20.rar
04-12
从github下载发行版的二进制包，手动部署每个组件，组成Kubernetes集群。 Kubeadm降低部署门槛，但屏蔽了很多细节，遇到问题很难排查。如果想更容易可控，推荐使用二进 制包部署Kubernetes集群，虽然手动部署麻烦点...
k8s高可用部署方案
weixin_44770684的博客
04-05 1512
k8s高可用部署方案
K8S高可用集群部署
weixin_44772835的博客
12-06 1353
准备环境 软件环境：软件版本操作系统CentOS7.8_x64 (mini)Docker20-ceKubernetes1.25服务器整体规划：角色IP其他单装组件k8s-master1192.168.1.71docker，etcd，nginx，keepalivedk8s-master2192.168.1.72docker，...
k8s高可用集群部署(二进制、containerd)
神神秘秘
12-11 1815
同步k8s更新后部署的坑
k8s高可用集群搭建部署
热门推荐
qq_40954652的博客
03-03 2万+
简介 k8s普通搭建出来只是单master节点，如果该节点挂掉，则整个集群都无法调度，K8s高可用集群是用多个master节点加负载均衡节点组成，外层再接高可用分布式存储集群例如ceph集群，实现计算能力+存储能力的高可用，同时，etcd也可以独立出来用外部的etcd集群 架构详解 高可用架构图 架构详解 访问链路步骤：外部域名访问 ——> node集群负载均衡 ——>ingress ——> node集群 ——> serivce ——> pod 内部元数据存储步骤：mas
Kubeadm部署高可用K8S集群(1)
2401_84411939的博客
04-19 378
外链图片转存中…(img-jscD6xRZ-1713460202408)][外链图片转存中…(img-EjXtnrAt-1713460202410)][外链图片转存中…(img-6Mf1nTJI-1713460202411)]
【kubernetes】k8s集群高可用部署安装和概念详细说明【含离线部署】，客户端连接haproxy访问高可用流程
2401_84048895的博客
04-18 868
又是一年求职季，在这里，我为各位准备了一套Java程序员精选高频面试笔试真题，来帮助大家攻下BAT的offer，题目范围从初级的Java基础到高级的分布式架构等等一系列的面试题和答案，用于给大家作为参考以下是部分内容截图《互联网大厂面试真题解析、进阶开发核心学习笔记、全套讲解视频、实战项目源码讲义》点击传送门即可获取！e]sudo chownid−uid−u执行环境变量创建【上面其实有提示的，自行看一下】可以直接复制你上面提示中的这3个命令哈id−uid−u。
Kubeadm方式部署K8s高可用集群
weixin_72583321的博客
05-17 1585
一个6节点的Kubernetes（K8s）高可用集群是一套由6台服务器组成的系统，旨在通过冗余和故障转移机制确保集群的稳定性和可靠性。它通常包括多个主节点（Master Nodes），这些节点运行着Kubernetes控制平面组件，如API服务器、调度器和控制器管理器，以及多个工作节点（Worker Nodes），它们负责运行容器化的应用。在高可用配置中，主节点会部署多个实例，并通过选举机制确保始终有一个主节点处于活跃状态，其他主节点则作为备份。工作节点则负责执行容器任务，并通过负载均衡和故障转移策略来保证
21、Kubernetes核心技术 - 高可用集群搭建（kubeadm+keepalived+haproxy）
weixiaohuai的博客
01-09 1114
前面我们介绍了使用kubeadm搭建k8s集群，当时只使用了一个master节点，其实是不满足k8s高可用的。因为当master节点发生宕机时，通过node节点将无法继续访问，所以整个集群将无法正常提供服务。本篇文章，我们将使用kubeadm，配合keepalived、haproxy来实现k8s集群的高可用。
【Kubernetes项目部署】k8s集群+高可用、负载均衡+防火墙
Mo_nor的博客
06-13 1268
负载均衡区域配置Keepalived+Nginx，实现负载均衡高可用，通过VIP 192.168.10.100和自定义的端口号即可访问K8S发布出来的服务。Pod使用hostPath类型的存储卷挂载，两个节点本地目录共享使用 /data，2个Pod副本测试页面自定义，但要不同，以做区分。在 Kubernetes 环境中，通过yaml文件的方式，创建2个Nginx Pod分别放置在两个不同的节点上；点击【添加】，选择【网络适配器】，点击【完成】；点击【虚拟机】，选择【设置】；#到两个node节点查看。
部署企业级K8s高可用集群（kubeadm-V1.21）详细教程
本文将详细介绍如何部署一套完整的K8s高可用集群（kubeadm-V1.21）。在生产环境中，部署Kubernetes集群通常有两种方式，一种是使用kubeadm工具，另一种是从GitHub下载二进制包手动部署。本文将采用kubeadm搭建集群的...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值