Centos7升级到Rocky8

BigPicture

由于Centos7的EOL问题，业内有很多需求需要将Centos7进行升级，CloudLinux开展的一项支持 RHEL 衍生产品主要版本之间迁移的计划的项目-ELevate 。 Leapp 实用程序 (opens new window) 和一些支持从非 RHEL 操作系统迁移的补丁 (opens new window) 用于执行升级。该工具可以在升级前进行相关的环境检查，如rpm相关依赖，内核模块的差异，不同版本间的模块差异等，最大程度上的减少升级风险，且由于是内网环境，无法之联外网，所以记录一下升级过程以及遇到的一些坑

操作步骤

在开始升级之前，需要对系统盘以及数据盘进行一次快照备份，避免出现问题无法回退

1. 解除文件锁，如果没有加锁，可以忽略

chattr -i /etc/sudoers
chattr -i /etc/sysctl.conf

2. 更新当前Centos到小版本最新

yum update -y
reboot

3. 安装elevate相关工具

yum install elevate-release-latest-el7.noarch.rpm 
cat > /etc/yum.repos.d/ELevate.repo << EOF
[elevate]
name=ELevate
baseurl= http://10.184.144.240/repo/elevate/el7/x86_64/
gpgcheck=1
enabled=1
priority=90
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-ELevate

[elevate-source]
name=name=ELevate - Source
baseurl = http://10.184.144.240/repo/elevate/el7/SRPMS/
gpgcheck=1
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-ELevate
EOF
yum clean all

4. 升级前检查

升级前需要进行一些关键配置的检查，比如当前内核情况，7和8 版本在部分系统服务上的差异，在升级完成后，可以在/var/log/leapp/leapp-report.txt进行报告检查

yum install -y leapp-upgrade leapp-data-rocky --nogpgcheck
leapp preupgrade --no-rhsm --enablerepo BaseOS --enablerepo AppStream
cat /var/log/leapp/leapp-report.txt

4.1. 检查项以及风险配置修改

4.1.1. 多内核冲突

Risk Factor: high (inhibitor)
Title: Multiple devel kernels installed
Summary: DNF cannot produce a valid upgrade transaction when multiple kernel-devel packages are installed.
Remediation: [hint] Remove all but one kernel-devel packages before running Leapp again.
[command] yum -y remove kernel-devel-3.10.0-1160.11.1.el7 kernel-devel-3.10.0-1160.71.1.el7 kernel-devel-3.10.0-1160.90.1.el7
Key: 8ceea81afbbb1a329b7d82ca7212b21509e5b256

以上报错主要是linux在升级了内核后，没有对已有的版本进行删除，删除重复版本即可,查看当前已安装kernel:

rpm -qa | grep kernel-devel

只保留最新的，其他删除

yum remove kernel-devel-3.10.0-1160.11.1.el7.x86_64 -y
yum remove kernel-devel-3.10.0-1160.15.2.el7.x86_64 -y
yum remove kernel-devel-3.10.0-1160.71.1.el7.x86_64 -y

4.1.2. pam_tally2模块弃用

Risk Factor: high (inhibitor)
Title: The pam_tally2 pam module(s) no longer available
Summary: The services login, sshd using PAM are configured to use pam_tally2 module(s), which is no longer available in Red Hat Enterprise Linux 8.
Remediation: [hint] If you depend on its functionality, it is recommended to migrate to pam_faillock. Otherwise please remove the pam module(s) from all the files under /etc/pam.d/.
Key: ce6abfb001da076686ffdd4ab61d28eb12d12569

rhel8开始会弃用老的pam_tally2.so的认证模块，所以需要把当前引用pam_tally2.so模块的地方修改下，注释掉pam模块的引用

cat /etc/pam.d/* | grep pam_tally2.so

注释掉这2行

sed -i 's/auth required pam_tally2.so/#&/g' /etc/pam.d/*

4.1.3. 提前配置answer file

Risk Factor: high (inhibitor)
Title: Missing required answers in the answer file
Summary: One or more sections in answerfile are missing user choices: remove_pam_pkcs11_module_check.confirm
For more information consult https://leapp.readthedocs.io/en/latest/dialogs.html
Remediation: [hint] Please register user choices with leapp answer cli command or by manually editing the answerfile.
[command] leapp answer --section remove_pam_pkcs11_module_check.confirm=True
Key: d35f6c6b1b1fa6924ef442e3670d90fa92f0d54b

在默认情况下，升级过程中是交互式的，Leapp 工具会生成 true 或 false 问题，必须手动回答。可以提前配置answer file 进行对应配置，如这里的提示是因为rhel8不再支持pam_pkcs11模块，所以在升级过程中会有是否Disable pam_pkcs11 module in PAM configuration的提示，这边提前确认即可。
leapp answer --section remove_pam_pkcs11_module_check.confirm=True

这一步需要在第一次preupgrade之前进行

4.1.4. python版本差别

Risk Factor: high
Title: Difference in Python versions and support in RHEL 8
Summary: In RHEL 8, there is no 'python' command. Python 3 (backward incompatible) is the primary Python version and Python 2 is available with limited support and limited set of packages. Read more here: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/configuring_basic_system_settings/#using-python3
Remediation: [hint] Please run "alternatives --set python /usr/bin/python3" after upgrade
Key: 0c98585b1d8d252eb540bf61560094f3495351f5

由于rhel8默认不在支持python2，所以需要在完成升级只切换python环境, 当然也可以不切换

4.1.5. Unable to install RHEL 8 userspace packages

2023-05-29 10:35:55.279196 [ERROR] Actor: target_userspace_creator
Message: Unable to install RHEL 8 userspace packages.
Summary:
    Details: Command ['systemd-nspawn', '--register=no', '--quiet', '-D', '/var/lib/leapp/scratch/mounts/root_/system_overlay', '--setenv=LEAPP_UPGRADE_PATH_TARGET_RELEASE=8.6', '--setenv=LEAPP_NO_RHSM=0', '--setenv=LEAPP_EXPERIMENTAL=0', '--setenv=LEAPP_UPGRADE_PATH_FLAVOUR=default', '--setenv=LEAPP_COMMON_TOOLS=:/etc/leapp/repos.d/system_upgrade/cloudlinux/tools:/etc/leapp/repos.d/system_upgrade/el7toel8/tools', '--setenv=LEAPP_COMMON_FILES=:/etc/leapp/repos.d/system_upgrade/common/files:/etc/leapp/repos.d/system_upgrade/el7toel8/files', '--setenv=LEAPP_IPU_IN_PROGRESS=7to8', '--setenv=LEAPP_UNSUPPORTED=0', '--setenv=LEAPP_EXECUTION_ID=727d2745-aad4-4535-ac69-7842c30a8d06', '--setenv=LEAPP_HOSTNAME=rocky-upgrade-test002', 'dnf', 'install', '-y', '--nogpgcheck', '--setopt=module_platform_id=platform:el8', '--setopt=keepcache=1', '--releasever', u'8.6', '--installroot', '/el8target', '--disablerepo', '*', '--enablerepo', u'rocky8-appstream', '--enablerepo', u'rocky8-powertools', '--enablerepo', u'rocky8-baseos', '--enablerepo', u'rocky8-resilient-storage', '--enablerepo', u'rocky8-ha', '--enablerepo', u'rocky8-extras', 'dnf', 'dnf-command(config-manager)', '--disableplugin', 'subscription-manager'] failed with exit code 1.
    Stderr: Failed to create directory /var/lib/leapp/scratch/mounts/root_/system_overlay//sys/fs/selinux: Read-only file system
            Failed to create directory /var/lib/leapp/scratch/mounts/root_/system_overlay//sys/fs/selinux: Read-only file system
            Host and machine ids are equal (20201228113502924739250506992733): refusing to link journals
            Failed to synchronize cache for repo 'rocky8-powertools', ignoring this repo.
            Failed to synchronize cache for repo 'rocky8-resilient-storage', ignoring this repo.
            Failed to synchronize cache for repo 'rocky8-ha', ignoring this repo.
            Failed to synchronize cache for repo 'rocky8-appstream', ignoring this repo.
            Failed to synchronize cache for repo 'rocky8-baseos', ignoring this repo.
            Failed to synchronize cache for repo 'rocky8-extras', ignoring this repo.
            Error: Unable to find a match: dnf dnf-command(config-manager)

在preupgrade的时候，会从源里进行一些软件包的/etc/leapp/files/leapp_upgrade_repositories.repo进行包的拉取，所以这里要配置一下repo源

cat > /etc/leapp/files/leapp_upgrade_repositories.repo << EOF 
[AppStream]
name=Rocky Linux $releasever - AppStream
baseurl=http://mirrors.cloud.aliyuncs.com/rockylinux/8/AppStream/\$basearch/os/
gpgcheck=1
enabled=1
countme=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial

[BaseOS]
name=Rocky Linux $releasever - BaseOS
baseurl=http://mirrors.cloud.aliyuncs.com/rockylinux/8/BaseOS/\$basearch/os/
gpgcheck=1
enabled=1
countme=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial
EOF

4.2. 包安装与卸载

为了避免一些冲突，建议在升级前把比如yum等包先行卸载，升级完成后再安装, 先安装dnf命令，后续升级先用dnf，再升级后重新安装yum使用

yum -y install rpmconf yum-utils dnf

卸载 yum manager

dnf -y remove yum yum-metadata-parser

临时打开ssh_root登录

echo PermitRootLogin yes | sudo tee -a /etc/ssh/sshd_config

5.升级

升级前记录当前os-release

cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

开始升级

leapp upgrade --no-rhsm --enablerepo BaseOS --enablerepo AppStream

升级过程大概在10分钟左右

reboot

第一次重启需要耗费相对较长的时间，建议可以在vnc端进行查看，大约在10-15分钟左右，第一次重启会在一个initramfs内，将el7的包，升级到el8。

6. 加固以及初始化配置

6.1. 配置repo

升级过程会在/etc/yum.repo.d 下生成默认的外网repo配置，需要清理掉

mkdir -p /etc/yum.repos.d/bak
mv /etc/yum.repos.d/* /etc/yum.repos.d/bak
cat /etc/leapp/files/leapp_upgrade_repositories.repo > /etc/yum.repos.d/rocky8.repo

6.2. 安全加固

由于部分安全配置在rhel7以及8中不一样了，需要单独处理
pam_tally2 在centos8已经不在使用了

echo 'auth required pam_faillocak.so preauth silent audit deny=5 unlock_time=180' >> /etc/pam.d/sshd

echo 'auth required pam_faillocak.so preauth silent audit deny=5 unlock_time=180 even_deny_root root_unlock_time=60'  >> /etc/pam.d/login

关闭ssh_root登录

echo PermitRootLogin no | sudo tee -a /etc/ssh/sshd_config
systemctl restart sshd

重新加载文件锁

chattr +i /etc/sudoers
chattr +i /etc/sysctl.conf

6.3. 修改当前python环境

考虑到部分业务可能使用了python2的代码，所以修改当前的python环境依然是python2

alternatives --set python /usr/bin/python2

6.4. 包安装

yum install yum yum-metadata-parser -y

el8版本的yum为4.7，避免了python版本的依赖问题，所以需要装回去，不要让用户修改习惯
6.5. 检查系统版本

cat /etc/os-release
NAME="Rocky Linux"
VERSION="8.8 (Green Obsidian)"
ID="rocky"
ID_LIKE="rhel centos fedora"
VERSION_ID="8.8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Rocky Linux 8.8 (Green Obsidian)"
ANSI_COLOR="0;32"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:rocky:rocky:8:GA"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
SUPPORT_END="2029-05-31"
ROCKY_SUPPORT_PRODUCT="Rocky-Linux-8"
ROCKY_SUPPORT_PRODUCT_VERSION="8.8"
REDHAT_SUPPORT_PRODUCT="Rocky Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.8"

7. 修改CMDB

升级完成后需要修改CMDB的操作系统栏，也可以批量升级后统一修改

8. 参考文档

https://wiki.almalinux.org/elevate/ELevate-quickstart-guide.html
https://access.redhat.com/solutions/5492401
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/upgrading_from_rhel_7_to_rhel_8/index