openEuler 20.03 (LTS-SP2) aarch64 cephadm 部署ceph18.2.0【1】离线部署 准备基础环境

准备3台虚拟机服务器(均可访问公网)

10.2.1.176 （作为操作机）

10.2.1.191

10.2.1.219

安装基础工具

yum install -y vim

配置hosts

编辑/etc/hosts，添加

10.2.1.176 ceph-176
10.2.1.191 ceph-191
10.2.1.219 ceph-219

配置免密登录（方便远程操作）

10.2.1.176 执行

[root@10-2-1-176 ~]# ssh-keygen
[root@10-2-1-176 ~]# ssh-copy-id ceph-191
[root@10-2-1-176 ~]# ssh-copy-id ceph-219

同步hosts

[root@10-2-1-176 ~]# scp /etc/hosts ceph-191:/etc/hosts 
[root@10-2-1-176 ~]# scp /etc/hosts ceph-219:/etc/hosts 

修改hostname（此处为修订内容，后续的章节可能于此不一致！修改量太大）

hostnamectl set-hostname ceph-176
ssh ceph-191 hostnamectl set-hostname ceph-191
ssh ceph-219 hostnamectl set-hostname ceph-219

配置ntp

内网服务端

[root@10-2-1-176 ~]# yum install -y ntp

[root@10-2-1-176 ~]# vim /etc/ntp.conf 
[root@10-2-1-176 ~]# systemctl enable ntpd
Created symlink /etc/systemd/system/multi-user.target.wants/ntpd.service → /usr/lib/systemd/system/ntpd.service.
[root@10-2-1-176 ~]# systemctl restart ntpd
[root@10-2-1-176 ~]# systemctl status ntpd
● ntpd.service - Network Time Service
   Loaded: loaded (/usr/lib/systemd/system/ntpd.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2023-12-04 15:15:40 CST; 8s ago
  Process: 22311 ExecStart=/usr/sbin/ntpd -u ntp:ntp $OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 22313 (ntpd)
    Tasks: 2
   Memory: 3.3M
   CGroup: /system.slice/ntpd.service
           └─22313 /usr/sbin/ntpd -u ntp:ntp -g

Dec 04 15:15:40 10-2-1-176 ntpd[22313]: Listen and drop on 0 v6wildcard [::]:123
Dec 04 15:15:40 10-2-1-176 ntpd[22313]: Listen and drop on 1 v4wildcard 0.0.0.0:123
Dec 04 15:15:40 10-2-1-176 ntpd[22313]: Listen normally on 2 lo 127.0.0.1:123
Dec 04 15:15:40 10-2-1-176 ntpd[22313]: Listen normally on 3 enp3s0 10.2.1.176:123
Dec 04 15:15:40 10-2-1-176 ntpd[22313]: Listen normally on 4 lo [::1]:123
Dec 04 15:15:40 10-2-1-176 ntpd[22313]: Listen normally on 5 enp3s0 [fe80::9372:4b38:76ef:92e4%2]:123
Dec 04 15:15:40 10-2-1-176 ntpd[22313]: Listening on routing socket on fd #22 for interface updates
Dec 04 15:15:40 10-2-1-176 ntpd[22313]: kernel reports TIME_ERROR: 0x2041: Clock Unsynchronized
Dec 04 15:15:40 10-2-1-176 ntpd[22313]: kernel reports TIME_ERROR: 0x2041: Clock Unsynchronized
Dec 04 15:15:40 10-2-1-176 systemd[1]: Started Network Time Service.
 

状态，目测正常

 查看防火墙状态

[root@10-2-1-176 ~]# systemctl status firewalld

开放udp123端口

安装工具，查看端口状态

yum install net-tools
netstat -lnep | grep ntp

开放端口

[root@10-2-1-176 ~]# firewall-cmd --permanent --add-port 123/udp 
success
[root@10-2-1-176 ~]# systemctl restart firewalld
[root@10-2-1-176 ~]# firewall-cmd --list-port
123/udp
 

内网客户端

安装ntp

[root@10-2-1-176 ~]# ssh ceph-191 yum install ntp -y
[root@10-2-1-176 ~]# ssh ceph-219 yum install ntp -y

测试内网时间服务器ip地址（成功）

[root@10-2-1-176 ~]# ssh ceph-191 ntpdate ceph-176

Authorized users only. All activities may be monitored and reported.
 4 Dec 15:32:19 ntpdate[22293]: adjust time server 10.2.1.176 offset +0.007469 sec

[root@10-2-1-176 ~]# ssh ceph-219 ntpdate ceph-176

Authorized users only. All activities may be monitored and reported.
 4 Dec 15:32:51 ntpdate[22327]: adjust time server 10.2.1.176 offset +0.009131 sec
 

主节点安装podman-3.4.4（编译rpm包）

参考官方适配的列表

COMPATIBILITY WITH PODMAN VERSIONS

Podman and Ceph have different end-of-life strategies. This means that care must be taken in finding a version of Podman that is compatible with Ceph.

This table shows which version pairs are expected to work or not work together:

Ceph	Podman
	1.9	2.0	2.1	2.2	3.0	> 3.0
<= 15.2.5	True	False	False	False	False	False
>= 15.2.6	True	True	True	False	False	False
>= 16.2.1	False	True	True	False	True	True
>= 17.2.0	False	True	True	False	True	True

选择版本> 3.0

自带的版本不满足

下载src.rpm(openEuler-22.03-LTS-SP2)

wget https://repo.openeuler.org/openEuler-22.03-LTS-SP2/source/Packages/podman-3.4.4-1.oe2203sp2.src.rpm

安装src.rpm

rpm -ivUh podman-3.4.4-1.oe2203sp2.src.rpm

安装依赖

yum-builddep -y ~/rpmbuild/SPECS/podman.spec

安装的依赖内容

构建rpm包

yum install rpm-build
rpmbuild -ba ~/rpmbuild/SPECS/podman.spec

podman 3.4.4 rpm包

构建crun 1.4.5 rpm包（podman3.4.4依赖）

wget https://repo.openeuler.org/openEuler-22.03-LTS-SP2/source/Packages/crun-1.4.5-1.oe2203sp2.src.rpm
rpm -ivUh crun-1.4.5-1.oe2203sp2.src.rpm
yum-builddep -y ~/rpmbuild/SPECS/crun.spec
rpmbuild -ba ~/rpmbuild/SPECS/crun.spec

crun 1.4.5 rpm包

构建conmon 2.1.0 rpm包（podman3.4.4依赖）

wget https://repo.openeuler.org/openEuler-22.03-LTS-SP2/source/Packages/conmon-2.1.0-1.oe2203sp2.src.rpm
rpm -ivUh conmon-2.1.0-1.oe2203sp2.src.rpm
yum-builddep -y rpmbuild/SPECS/conmon.spec 
rpmbuild -ba rpmbuild/SPECS/conmon.spec

common 2.1.0 rpmb包

主节点安装podman

yum install -y rpmbuild/RPMS/noarch/podman-docker-3.4.4-1.noarch.rpm rpmbuild/RPMS/aarch64/podman-remote-3.4.4-1.aarch64.rpm rpmbuild/RPMS/aarch64/podman-3.4.4-1.aarch64.rpm rpmbuild/RPMS/aarch64/crun-help-1.4.5-1.aarch64.rpm rpmbuild/RPMS/aarch64/crun-1.4.5-1.aarch64.rpm rpmbuild/RPMS/aarch64/conmon-2.1.0-1.aarch64.rpm rpmbuild/RPMS/aarch64/podman-help-3.4.4-1.aarch64.rpm rpmbuild/RPMS/aarch64/podman-gvproxy-3.4.4-1.aarch64.rpm rpmbuild/RPMS/aarch64/podman-plugins-3.4.4-1.aarch64.rpm

启动podman

[root@10-2-1-176 ~]# systemctl daemon-reload
[root@10-2-1-176 ~]# systemctl enable podman
[root@10-2-1-176 ~]# systemctl start podman
[root@10-2-1-176 ~]# systemctl status podman

其他节点安装podman

同步rpm包

yum install -y rsync
ssh ceph-191 yum install -y rsync
ssh ceph-219 yum install -y rsync
rsync -avr -P ~/rpmbuild/RPMS ceph-191:~/rpmbuild/
rsync -avr -P ~/rpmbuild/RPMS ceph-219:~/rpmbuild/

安装podman 3.4.4

ssh ceph-191 yum install -y rpmbuild/RPMS/noarch/podman-docker-3.4.4-1.noarch.rpm rpmbuild/RPMS/aarch64/podman-remote-3.4.4-1.aarch64.rpm rpmbuild/RPMS/aarch64/podman-3.4.4-1.aarch64.rpm rpmbuild/RPMS/aarch64/crun-help-1.4.5-1.aarch64.rpm rpmbuild/RPMS/aarch64/crun-1.4.5-1.aarch64.rpm rpmbuild/RPMS/aarch64/conmon-2.1.0-1.aarch64.rpm rpmbuild/RPMS/aarch64/podman-help-3.4.4-1.aarch64.rpm rpmbuild/RPMS/aarch64/podman-gvproxy-3.4.4-1.aarch64.rpm rpmbuild/RPMS/aarch64/podman-plugins-3.4.4-1.aarch64.rpm

ssh ceph-219 yum install -y rpmbuild/RPMS/noarch/podman-docker-3.4.4-1.noarch.rpm rpmbuild/RPMS/aarch64/podman-remote-3.4.4-1.aarch64.rpm rpmbuild/RPMS/aarch64/podman-3.4.4-1.aarch64.rpm rpmbuild/RPMS/aarch64/crun-help-1.4.5-1.aarch64.rpm rpmbuild/RPMS/aarch64/crun-1.4.5-1.aarch64.rpm rpmbuild/RPMS/aarch64/conmon-2.1.0-1.aarch64.rpm rpmbuild/RPMS/aarch64/podman-help-3.4.4-1.aarch64.rpm rpmbuild/RPMS/aarch64/podman-gvproxy-3.4.4-1.aarch64.rpm rpmbuild/RPMS/aarch64/podman-plugins-3.4.4-1.aarch64.rpm

启动podman

ssh ceph-191 systemctl daemon-reload
ssh ceph-191 systemctl enable podman
ssh ceph-191 systemctl start podman
ssh ceph-191 systemctl status podman

ssh ceph-219 systemctl daemon-reload
ssh ceph-219 systemctl enable podman
ssh ceph-219 systemctl start podman
ssh ceph-219 systemctl status podman

至此，基础环境准备就绪。

接下来

搭建registry私服

拉取ceph镜像，导入私服

配置各节点insecury-registry

根据ceph镜像内部的cephadm拉取所有镜像，导入私服

利用私服boostrap主节点

配置全部采用私服镜像

添加、启动其他节点

添加mon

添加osd

。。。

参考 

Compatibility and Stability — Ceph Documentation

openEuler 22.03 LTS x86_64 cephadm 部署ceph 16.2.14 未完成 笔记-CSDN博客