【kubernetes】k8s集群高可用部署安装和概念详细说明【含离线部署】，客户端连接haproxy访问高可用流程

[root@ccx haproxy]# yum -y install etcd --downloadonly --downloaddir=/root/etcd

Loaded plugins: fastestmirror, langpacks

Loading mirror speeds from cached hostfile

• base: mirror.lzu.edu.cn

• extras: mirrors.aliyun.com

• updates: mirrors.aliyun.com

Resolving Dependencies

–> Running transaction check

—> Package etcd.x86_64 0:3.3.11-2.el7.centos will be installed

–> Finished Dependency Resolution

Dependencies Resolved

==========================================================================================

Package Arch Version Repository Size

==========================================================================================

Installing:

etcd x86_64 3.3.11-2.el7.centos extras 10 M

Transaction Summary

==========================================================================================

Install 1 Package

Total download size: 10 M

Installed size: 45 M

Background downloading packages, then exiting:

warning: /root/etcd/etcd-3.3.11-2.el7.centos.x86_64.rpm.55125.tmp: Header V3 RSA/SHA256 Si

gnature, key ID f4a80eb5: NOKEYPublic key for etcd-3.3.11-2.el7.centos.x86_64.rpm.55125.tmp is not installed

etcd-3.3.11-2.el7.centos.x86_64.rpm | 10 MB 00:07:24

exiting because “Download Only” specified

[root@ccx haproxy]# cd /root/etcd/

[root@ccx etcd]# ls

etcd-3.3.11-2.el7.centos.x86_64.rpm

[root@ccx etcd]#

• 然后导入内网并安装

两台etcd均需要安装

[root@etcd-161 etcd]# ls

etcd-3.3.11-2.el7.centos.x86_64.rpm

[root@etcd-161 etcd]#

[root@etcd-161 etcd]# rpm -ivhU * --nodeps --force

准备中… ################################# [100%]

正在升级/安装…

1:etcd-3.3.11-2.el7.centos ################################# [100%]

[root@etcd-161 etcd]#

[root@etcd-161 etcd]# scp etcd-3.3.11-2.el7.centos.x86_64.rpm 192.168.59.160:~

The authenticity of host ‘192.168.59.160 (192.168.59.160)’ can’t be established.

ECDSA key fingerprint is SHA256:zRtVBoNePoRXh9aA8eppKwwduS9Rjjr/kT5a7zijzjE.

ECDSA key fingerprint is MD5:b8:53:cc:da:86:2a:97:dc:bd:64:6b:b1:d0:f3:02:ce.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added ‘192.168.59.160’ (ECDSA) to the list of known hosts.

root@192.168.59.160’s password:

Permission denied, please try again.

root@192.168.59.160’s password:

etcd-3.3.11-2.el7.centos.x86_64.rpm 100% 10MB 38.8MB/s 00:00

[root@etcd-161 etcd]#

#另一台

[root@etcd-160 ~]# mkdir etcd

[root@etcd-160 ~]# mv etcd-3.3.11-2.el7.centos.x86_64.rpm etcd

[root@etcd-160 ~]# cd etcd

[root@etcd-160 etcd]# ls

etcd-3.3.11-2.el7.centos.x86_64.rpm

[root@etcd-160 etcd]#

[root@etcd-160 etcd]# rpm -ivhU * --nodeps --force

准备中… ################################# [100%]

正在升级/安装…

1:etcd-3.3.11-2.el7.centos ################################# [100%]

[root@etcd-160 etcd]#

编辑配置文件

• 两台都需要编辑，注意看主机名【需要对应修改ip】

看不懂的去我之前对etcd的安装说明博客，里面有详细介绍，我这就不做说明了

k8s的核心组件etcd的安装使用、快照说明及etcd命令详解【含单节点，多节点和新节点加入说明】

• 编辑配置文件

记得修改ip和ETCD_NAME行

[root@etcd-161 ~]# ip a | grep 59

inet 192.168.59.161/24 brd 192.168.59.255 scope global ens32

[root@etcd-161 ~]#

[root@etcd-161 ~]# cat /etc/etcd/etcd.conf

ETCD_DATA_DIR=“/var/lib/etcd/cluster.etcd”

ETCD_LISTEN_PEER_URLS=“http://192.168.59.161:2380,http://localhost:2380”

ETCD_LISTEN_CLIENT_URLS=“http://192.168.59.161:2379,http://localhost:2379”

ETCD_NAME=“etcd-161”

ETCD_INITIAL_ADVERTISE_PEER_URLS=“http://192.168.59.161:2380”

ETCD_ADVERTISE_CLIENT_URLS=“http://localhost:2379,http://192.168.59.161:2379”

ETCD_INITIAL_CLUSTER=“etcd-161=http://192.168.59.161:2380,etcd-160=http://192.168.59.160:2380”

ETCD_INITIAL_CLUSTER_TOKEN=“etcd-cluster”

ETCD_INITIAL_CLUSTER_STATE=“new”

[root@etcd-161 ~]#

另一台

[root@etcd-160 etcd]# ip a | grep 59

inet 192.168.59.160/24 brd 192.168.59.255 scope global ens32

[root@etcd-160 etcd]#

[root@etcd-160 etcd]# cat /etc/etcd/etcd.conf

ETCD_DATA_DIR=“/var/lib/etcd/cluster.etcd”

ETCD_LISTEN_PEER_URLS=“http://192.168.59.160:2380,http://localhost:2380”

ETCD_LISTEN_CLIENT_URLS=“http://192.168.59.160:2379,http://localhost:2379”

ETCD_NAME=“etcd-160”

ETCD_INITIAL_ADVERTISE_PEER_URLS=“http://192.168.59.160:2380”

ETCD_ADVERTISE_CLIENT_URLS=“http://localhost:2379,http://192.168.59.160:2379”

ETCD_INITIAL_CLUSTER=“etcd-161=http://192.168.59.161:2380,etcd-160=http://192.168.59.160:2380”

ETCD_INITIAL_CLUSTER_TOKEN=“etcd-cluster”

ETCD_INITIAL_CLUSTER_STATE=“new”

[root@etcd-160 etcd]#

• 然后启动etcd服务

[root@etcd-161 ~]# systemctl start etcd

[root@etcd-161 ~]# systemctl is-active etcd

active

[root@etcd-161 ~]# systemctl enable etcd

Created symlink from /etc/systemd/system/multi-user.target.wants/etcd.service to /usr/lib/systemd/system/etcd.service.

[root@etcd-161 ~]#

[root@etcd-160 etcd]# systemctl start etcd

[root@etcd-160 etcd]# systemctl is-active etcd

active

[root@etcd-160 etcd]# systemctl enable etcd

Created symlink from /etc/systemd/system/multi-user.target.wants/etcd.service to /usr/lib/systemd/system/etcd.service.

[root@etcd-160 etcd]#

master和work配置【集群配置】

---

环境配置【master和work都做】

• 这个呢，如果对集群配置不熟悉，去看看这篇文章

【kubernetes】k8s集群的搭建安装详细说明【创建集群、加入集群、踢出集群、重置集群…】【含离线搭建方法】

• 解析设置

master和node节点解析配置成一致，且互相之间都需要加上。

[root@master1-163 ~]# cat /etc/hosts

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4

::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.59.163 master1-163

192.168.59.162 master2-162

192.168.59.165 worker-165

[root@master1-163 ~]#

[root@master1-163 ~]# scp /etc/hosts 192.168.59.162:/etc/hosts

The authenticity of host ‘192.168.59.162 (192.168.59.162)’ can’t be established.

ECDSA key fingerprint is SHA256:zRtVBoNePoRXh9aA8eppKwwduS9Rjjr/kT5a7zijzjE.

ECDSA key fingerprint is MD5:b8:53:cc:da:86:2a:97:dc:bd:64:6b:b1:d0:f3:02:ce.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added ‘192.168.59.162’ (ECDSA) to the list of known hosts.

root@192.168.59.162’s password:

hosts 100% 238 297.8KB/s 00:00

[root@master1-163 ~]#

[root@master1-163 ~]# scp /etc/hosts 192.168.59.165:/etc/hosts

The authenticity of host ‘192.168.59.165 (192.168.59.165)’ can’t be established.

ECDSA key fingerprint is SHA256:zRtVBoNePoRXh9aA8eppKwwduS9Rjjr/kT5a7zijzjE.

ECDSA key fingerprint is MD5:b8:53:cc:da:86:2a:97:dc:bd:64:6b:b1:d0:f3:02:ce.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added ‘192.168.59.165’ (ECDSA) to the list of known hosts.

root@192.168.59.165’s password:

hosts 100% 238 294.3KB/s 00:00

[root@master1-163 ~]#

• 关闭swap

全部都要做

[root@master1-163 ~]# swapoff -a ; sed -i ‘/swap/d’ /etc/fstab

[root@master1-163 ~]#

[root@master1-163 ~]# swapon -s

[root@worker-165 ~]# swapon -s

文件名 类型 大小 已用 权限

/dev/sda2 partition 10485756 0 -1

[root@worker-165 ~]#

[root@worker-165 ~]# swapoff -a ; sed -i ‘/swap/d’ /etc/fstab

[root@worker-165 ~]# swapon -s

[root@worker-165 ~]#

[root@master2-162 ~]# swapon -s

文件名 类型 大小 已用 权限

/dev/sda2 partition 10485756 0 -1

[root@master2-162 ~]# swapoff -a ; sed -i ‘/swap/d’ /etc/fstab

[root@master2-162 ~]#

[root@master2-162 ~]# swapon -s

[root@master2-162 ~]#

• 关闭防火墙

master和node都需要执行

[root@master1-163 ~]# systemctl stop firewalld.service ; systemctl disable firewalld.service

Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.

Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

[root@master1-163 ~]#

• 关闭selinux

master和node都需要执行

[root@master2-162 docker-ce]# cat /etc/sysconfig/selinux | grep dis

disabled - No SELinux policy is loaded.

SELINUX=disabled

[root@master2-162 docker-ce]#

[root@master2-162 docker-ce]# getenforce

Disabled

[root@master2-162 docker-ce]#

• 配置加速器

master和node都需要执行

[root@master2-162 docker-ce]# cat > /etc/docker/daemon.json <<EOF

{

“registry-mirrors”: [“https://frz7i079.mirror.aliyuncs.com”]

}

EOF

[root@master2-162 docker-ce]# systemctl restart docker

[root@master2-162 docker-ce]#

• 设置内核参数

master和node都需要执行

[root@worker-165 docker-ce]# cat < /etc/sysctl.d/k8s.conf

net.bridge.bridge-nf-call-ip6tables = 1

net.bridge.bridge-nf-call-iptables = 1

net.ipv4.ip_forward = 1

EOF

[root@worker-165 docker-ce]#

[root@worker-165 docker-ce]# sysctl -p /etc/sysctl.d/k8s.conf

net.bridge.bridge-nf-call-ip6tables = 1

net.bridge.bridge-nf-call-iptables = 1

net.ipv4.ip_forward = 1

[root@worker-165 docker-ce]#

安装docker-ce【master和work都安装】

• 需要先配置一个yum源

[root@ccx etcd]# wget ftp://ftp.rhce.cc/k8s/* -P /etc/yum.repos.d/

–2021-11-26 16:47:07-- ftp://ftp.rhce.cc/k8s/*

=> ‘/etc/yum.repos.d/.listing’

Resolving ftp.rhce.cc (ftp.rhce.cc)… 101.37.152.41

…

• 如果有外网，直接执行

• 没有外网的，