openshift集群部署

完颜振江

已于 2023-09-21 17:47:37 修改

阅读量349

点赞数 2

文章标签： eureka 云原生

于 2023-06-30 17:58:02 首次发布

该文详细描述了OpenShift集群的安装过程，包括服务器配置、依赖包安装、Docker存储设置、节点间SSH信任、etcd集群搭建、OpenShiftAnsible安装以及安装后的验证，如节点状态和etcd健康检查。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

1.1 服务器基本信息

　　本次安装采用一个master、5个node、3个etcd，node节点两块硬盘，60G磁盘用于docker storage，xxx改为自己的域名或主机名

节点/主机名	功能	IP	内存	磁盘	CPU
master1.xxx.net	Master节点	192.168.10.110	16G	40G	8C
node1.xxx.net	Node节点	192.168.10.112	8G	40G/60G	4C
node2.xxx.net	Node节点	192.168.10.113	8G	40G/60G	4C
node3.xxx.net	Node节点	192.168.10.116	8G	40G/60G	4C
node4.xxx.net	Node节点	192.168.10.114	8G	40G/60G	4C
node5.xxx.net	Node节点	192.168.10.117	8G	40G/60G	4C
etcd1.xxx.net	etcd	192.168.10.109	4G	40G	2C
etcd2.xxx.net	etcd	192.168.10.111	4G	40G	2C
etcd3.xxx.net	etcd	192.168.10.115	4G	40G	2C

1.2 设置hosts信息+设置ssh信任关系+网络正常+服务器时间最新+安装最新版本的docker软件

1.3安装基础依赖包

yum install wget git net-tools bind-utils iptables-services bridge-utils bash-completion kexec-tools sos psacct vim ntpdate httpd-tools -y

1.4 所有Node节点执行docker-storage-setup

　　　[root@openshift-node1 ~]# docker-storage-setup  

 　　 WARNING: Device for PV 28oz2p-ZKrx-gSc2-k6Tg-E49Y-MK4A-YcQq7h not found or rejected by a filter.

 　　 WARNING: Device for PV 28oz2p-ZKrx-gSc2-k6Tg-E49Y-MK4A-YcQq7h not found or rejected by a filter.

  　　INFO: Device node /dev/sdb1 exists.

 　　 WARNING: Device for PV 28oz2p-ZKrx-gSc2-k6Tg-E49Y-MK4A-YcQq7h not found or rejected by a filter.

  　　Physical volume "/dev/sdb1" successfully created.

  　　WARNING: Device for PV 28oz2p-ZKrx-gSc2-k6Tg-E49Y-MK4A-YcQq7h not found or rejected by a filter.

  　　Volume group "docker-vg" successfully created

  　　WARNING: Device for PV 28oz2p-ZKrx-gSc2-k6Tg-E49Y-MK4A-YcQq7h not found or rejected by a filter.

  　　Using default stripesize 64.00 KiB.

  　　Rounding up size to full physical extent 84.00 MiB

  　　Thin pool volume with chunk size 512.00 KiB can address at most 126.50 TiB of data.

  　　Logical volume "docker-pool" created.

  　　Logical volume docker-vg/docker-pool changed.

1.5 所有节点更改docker仓库地址

　　cat /etc/sysconfig/docker

　　# /etc/sysconfig/docker

　　# Modify these options if you want to change the way the docker daemon runs

　　# OPTIONS='--selinux-enabled --log-driver=journald --signature-verification=false'

　　OPTIONS='--selinux-enabled --log-driver=journald --registry-mirror=https://docker.mirrors.ustc.edu.cn'

1.6 Master节点更改epel源,并安装ansible

　　yum -y install https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-7-11.noarch.rpm

　　sed -i -e "s/^enabled=1/enabled=0/" /etc/yum.repos.d/epel.repo

　　yum -y --enablerepo=epel install ansible pyOpenSSL

1.7 etcd集群安装

　　安装etcd集群（按需安装，不与openshift在同一集群）

　　所有节点关闭firewalld

　　[root@etcd1 ~]# systemctl stop firewalld

　　[root@etcd1 ~]# systemctl disable firewalld

　　Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.

　　Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

所有etcd节点开启iptables

　　[root@etcd1 ~]# systemctl start iptables

　　[root@etcd1 ~]# systemctl enable iptables

　　Created symlink from /etc/systemd/system/basic.target.wants/iptables.service to /usr/lib/systemd/system/iptables.service

所有etcd节点安装etcd，OpenShift高级安装模式无需自行配置etcd

yum install etcd -y

1.8 OpenShift高级安装

　　参考文档：https://docs.openshift.org/latest/install_config/install/advanced_install.html　

　　在master1节点上

　　[root@master1 ~]# cat /etc/ansible/hosts

　　# Create an OSEv3 group that contains the masters, nodes, and etcd groups

　　[OSEv3:children]

　　masters

　　nodes

　　etcd

　　#lb

　　# Set variables common for all OSEv3 hosts

　　[OSEv3:vars]

　　# SSH user, this user should allow ssh based auth without requiring a password

　　ansible_ssh_user=root

　　ansible_become=yes

　　debug_level=2

　　openshift_deployment_type=origin

　　# If ansible_ssh_user is not root, ansible_become must be set to true

　　#ansible_become=true

　　openshift_repos_enable_testing=true

　　openshift_enable_service_catalog=false

　　template_service_broker_install=false

　　# uncomment the following to enable htpasswd authentication; defaults to DenyAllPasswordIdentityProvider

　　openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider', 'filename': '/etc/origin/master/htpasswd'}]

　　openshift_disable_check=disk_availability,docker_storage,memory_availability,docker_image_availability

　　# config for metrics

　　openshift_release=3.6.1

　　openshift_clock_enabled=true

　　#openshift_master_cluster_method=native

　　#openshift_master_cluster_hostname=openshift.xxx.net

　　#openshift_master_cluster_public_hostname=openshift.xxx.net

　　#openshift_node_kubelet_args={'pods-per-core': ['10'], 'max-pods': ['250'], 'image-gc-high-threshold': ['90'], 'image-gc-low-threshold': ['80']}

　　# host group for masters

　　[masters]

　　master1.xxx.net

　　#master2.xxx.net

　　# host group for lb

　　#[lb]

　　#lb.xxx.net

　　# host group for etcd

　　[etcd]

　　etcd3.xxx.net

　　etcd5.xxx.net

　　etcd4.xxx.net

　　# host group for nodes, includes region info

　　[nodes]

　　master1.xxx.net

　　#master2.xxx.net

　　node1.xxx.net

　　node2.xxx.net openshift_node_labels="{'region': 'infra', 'zone': 'default'}"

　　node3.xxx.net openshift_node_labels="{'region': 'infra', 'zone': 'default'}"

　　node4.xxx.net

　　node5.xxx.net openshift_node_labels="{'region': 'infra', 'zone': 'default'}"

下载openshift-ansible，一般安装什么版本的openshift，就下载对应的tar包，但是要修改hosts文件对应的版本openshift_release=3.6.1：

wget https://github.com/openshift/openshift-ansible/archive/openshift-ansible-3.6.173.0.104-1.tar.gz

解压并执行安装(安装之前可以把镜像提前下载下来，避免因为网络问题导致安装失败)：

ansible-playbook -i /etc/ansible/hosts openshift-ansible-openshift-ansible-3.6.173.0.104-1/playbooks/byo/config.yml

1.9 验证安装　

　　Master1节点上验证node

　　[root@master1 ~]# oc get nodes

　　NAME                        STATUS                     AGE       VERSION

　　master1.xxx.net   Ready,SchedulingDisabled   38m       v1.6.1+5115d708d7

　　node1.xxx.net     Ready                      38m       v1.6.1+5115d708d7

　　node2.xxx.net     Ready                      38m       v1.6.1+5115d708d7

　　node3.xxx.net     Ready                      38m       v1.6.1+5115d708d7

　　node4.xxx.net     Ready                      38m       v1.6.1+5115d708d7

　　node5.xxx.net     Ready                      38m       v1.6.1+5115d708d7
复制代码

　Master1节点上验证etcd

　　[root@master1 ~]# yum install etcd -y

　　[root@master1 ~]# etcdctl -C     https://etcd1.xxx.net:2379,https://etcd3.xxx.net:2379,https://etcd2.xxx.net:2379     --ca-file=/etc/origin/master/master.etcd-ca.crt     --cert-file=/etc/origin/master/master.etcd-client.crt     --key-file=/etc/origin/master/master.etcd-client.key cluster-health

　　member 17c82e7e21b639e7 is healthy: got healthy result from https://192.168.10.109:2379

　　member 3bd39337b17b1a4e is healthy: got healthy result from https://192.168.10.111:2379

　　member 62cacf31d21cfcd4 is healthy: got healthy result from https://192.168.10.115:2379

　　cluster is healthy

　　[root@master1 ~]# etcdctl -C     https://etcd1.xxx.net:2379,https://etcd3.xxx.net:2379,https://etcd2.xxx.net:2379     --ca-file=/etc/origin/master/master.etcd-ca.crt     --cert-file=/etc/origin/master/master.etcd-client.crt     --key-　　file=/etc/origin/master/master.etcd-client.key member list

　　17c82e7e21b639e7: name=etcd1.xxx.net peerURLs=https://192.168.10.109:2380 clientURLs=https://192.168.10.109:2379 isLeader=false

　　3bd39337b17b1a4e: name=etcd2.xxx.net peerURLs=https://192.168.10.111:2380 clientURLs=https://192.168.10.111:2379 isLeader=false

　　62cacf31d21cfcd4: name=etcd3.xxx.net peerURLs=https://192.168.10.115:2380 clientURLs=https://192.168.10.115:2379 isLeader=true

1.11 访问控制台

　　高级安装模式下会安装router(安装在infra节点上)、registry、console，均可直接使用。

　　创建控制台账号Master节点

　　htpasswd -b /etc/origin/master/htpasswd dev dev2018

　　登录控制台：https://master1.xxx.net:8443

　　此地址需要解析到master主机上