NSX ALB + Harbor + OpenShift 4.8 UPI安装配置实验笔记系列目录
目录
3 基于本地Registry的Image解压openshift-install安装程序
4.2 使用变量生成install-config.yaml文件
1 声明基础临时环境变量
因OpenShift会有版本更新情况,当非连续性操作时,可在此使用实际我们上面Download下来的RELEASE版本号更从新声明变量:
export OCP_RELEASE=4.8.36
export RHCOS_RELEASE=4.8.14
export LOCAL_REGISTRY='map.corp.tanzu'
export LOCAL_REPOSITORY='openshift/ocp4.8.36'
export PRODUCT_REPO='openshift-release-dev'
export RELEASE_NAME='ocp-release'
export ARCHITECTURE='x86_64'
export OCP_PATH=/data/OCP-${OCP_RELEASE}/ocp
export LOCAL_SECRET_JSON=${OCP_PATH}/secret/pull-secret.json
export REMOVABLE_MEDIA_PATH=${OCP_PATH}/ocp-image
export BOOT_FILE_PATH=/data/boot-files
export RHCOS_ISO_PATH=${BOOT_FILE_PATH}/rhcos-iso
export DOMAIN=corp.tanzu
export OCP_CLUSTER_ID=ocp
export OPERATOR_DOMAIN=operator.${DOMAIN}
export IGN_PATH=${BOOT_FILE_PATH}/ignition/${OCP_CLUSTER_ID}
2 为RHCOS的core用户准备ssh key
1). 在安装文件目录“/data/boot-files/ignition/ocp”建个ssh-key目录,用于存放我们生成的ssh key:
mkdir -p ${IGN_PATH}/ssh-key
ssh-keygen -t rsa -b 4096 -N '' -f ${IGN_PATH}/ssh-key/id_rsa
eval "$(ssh-agent -s)"
ssh-add ${IGN_PATH}/ssh-key/id_rsa
2). 为生成的ssh key声明环境变量:
export SSH_PRI_FILE=${IGN_PATH}/ssh-key/id_rsa
export SSH_PUB_STR=$(cat ${IGN_PATH}/ssh-key/id_rsa.pub)
3 基于本地Registry的Image解压openshift-install安装程序
oc adm release extract -a ${LOCAL_SECRET_JSON} --command=openshift-install "${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE}-${ARCHITECTURE}"
cp openshift-install /usr/sbin/
openshift-install version
4 生成install-config.yaml文件
4.1 准备本地Registry证书
将本地Registry的CA证书copy至准备目录:
cp /etc/pki/ca-trust/source/anchors/map-harbor.crt ${IGN_PATH}
4.2 使用变量生成install-config.yaml文件
1). 在Operator主机输入以下命令信息,此命令使用变量在yaml文件中同时更新了PullSecret、sshKey、本地Registry的ca证书、还有本地Registry的域名等信息:
cat << EOF > ${IGN_PATH}/install-config.yaml
apiVersion: v1
baseDomain: ${DOMAIN}
compute:
- hyperthreading: Enabled
name: worker
replicas: 0
controlPlane:
hyperthreading: Enabled
name: master
replicas: 3
metadata:
name: ${OCP_CLUSTER_ID}
networking:
clusterNetworks:
- cidr: 100.224.0.0/16
hostPrefix: 24
networkType: OpenShiftSDN
serviceNetwork:
- 100.225.0.0/16
platform:
none: {}
fips: false
pullSecret: '$(awk -v RS= '{$1=$1}1' ${LOCAL_SECRET_JSON})'
sshKey: '${SSH_PUB_STR}'
additionalTrustBundle: |
$(cat ${IGN_PATH}/map-harbor.crt | sed 's/^/ /g' | sed 's/^/ /g')
imageContentSources:
- mirrors:
- ${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}
source: quay.io/openshift-release-dev/ocp-release
- mirrors:
- ${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}
source: quay.io/openshift-release-dev/ocp-v4.0-art-dev
EOF
2). 完成后使用以下命令查看一下yaml文件的内容:
more ${IGN_PATH}/install-config.yaml
3). 确认没有问题后,可以将其备份一份,因为生成ignition过程中此yaml会被销毁:
cp ${IGN_PATH}/install-config.yaml{,.`date '+%s'`.bak}
5 生成ignition文件
注:创建完ignition文件后,必须在24小时内完成OpenShift集群的创建,否则证书会过期失效。
5.1 启用Operator主机的http服务
yum -y install httpd
systemctl enable httpd
systemctl start httpd
systemctl status httpd
5.2 创建mainfest文件
1). 创建mainfest文件
openshift-install create manifests --dir ${IGN_PATH}
2). 查看成生成的结果
tree ${IGN_PATH}/manifests/ ${IGN_PATH}/openshift/
3). 修改Master节点不参与业务POD调度配置(可选 )
sed -i 's/mastersSchedulable: true/mastersSchedulable: false/g' ${IGN_PATH}/manifests/cluster-scheduler-02-config.yml
4). 检查配置结果
cat ${IGN_PATH}/manifests/cluster-scheduler-02-config.yml
5.3 按需修改配置文件
5.3.1 修改应用默认域名
Openshift应用默认子域名为“apps”,如需要调,可在此时修改cluster-ingress-02-config.yml内的参数,此ymal文件内容如下:
修改命令如下,修改域名为dev.ocp.corp.tanzu:
sed -i 's/apps.ocp/dev.ocp/g' ${IGN_PATH}/manifests/cluster-ingress-02-config.yml
5.3.2 配置NTP
1). 分别为worker和master生成chrony.bu文件,具体如下:
Worker:
cat << EOF > 99-worker-chrony.bu
variant: openshift
version: 4.8.0
metadata:
name: 99-worker-chrony
labels:
machineconfiguration.openshift.io/role: worker
storage:
files:
- path: /etc/chrony.conf
mode: 0644
overwrite: true
contents:
inline: |
pool 192.168.100.1 iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
EOF
Master:
cat << EOF > 99-master-chrony.bu
variant: openshift
version: 4.8.0
metadata:
name: 99-master-chrony
labels:
machineconfiguration.openshift.io/role: master
storage:
files:
- path: /etc/chrony.conf
mode: 0644
overwrite: true
contents:
inline: |
pool 192.168.100.1 iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
EOF
2). 生成对应的yaml文件,并存入安装路径下的openshift目录中:
butane 99-worker-chrony.bu -o ${IGN_PATH}/openshift/99-worker-chrony.yaml
butane 99-master-chrony.bu -o ${IGN_PATH}/openshift/99-master-chrony.yaml
3). 如果环境已安装,则可以用以下命令应用:
oc apply -f 99-worker-chrony.yaml
oc apply -f 99-master-chrony.yaml
5.4 生成Ignition文件
在我们安装目录中生成ignition文件并查看生成结果:
openshift-install create ignition-configs --dir ${IGN_PATH}/
ls -al ${IGN_PATH}/*.ign
5.5 创建Ignition引导文件http下载目录
1). 给安装文件目录配置权限:
chmod -R 755 ${IGN_PATH}
chmod -R 600 ${IGN_PATH}/ssh-key
注:如果ssh-key目录或文件权限过大,在ssh时会有“Permissions 0755 for '/data/boot-files/ignition/ocp/ssh-key/id_rsa' are too open.”报错提示。
2). 配置httpd config:
cat << EOF > /etc/httpd/conf.d/ignition.conf
Alias /ignition "${IGN_PATH}/../"
<Directory "${IGN_PATH}/../">
Options +Indexes +FollowSymLinks
Require all granted
</Directory>
<Location /ignition >
SetHandler None
</Location>
EOF
3). 重启http服务:
systemctl restart httpd
4). 检查此http服务是否正常
curl http://${OPERATOR_DOMAIN}/ignition/ocp/