本系列文章包含从OpenStack离线源制作到完成OpenStack安装的全部过程。
在本系列教程中使用的OpenStack的安装版本为第20个版本Train(简称T版本),2020年5月13日,OpenStack社区发布了第21个版本Ussuri(简称U版本)。
OpenStack部署系列文章
OpenStack Victoria版 安装部署系列教程
OpenStack Ussuri版 离线安装部署系列教程(全)
OpenStack Train版 离线安装部署系列教程(全)
欢迎留言沟通,共同进步。
文章目录
11.2存储节点-Cinder存储服务组件
在存储节点服务器安装cinder存储服务
官方参考:
https://docs.openstack.org/install-guide/openstack-services.html
https://docs.openstack.org/train/install/
https://docs.openstack.org/cinder/train/install/
https://docs.openstack.org/cinder/train/install/index-rdo.html
https://docs.openstack.org/cinder/train/install/cinder-storage-install-rdo.html
(可选服务)
https://docs.openstack.org/cinder/train/install/cinder-backup-install-rdo.html
存储节点,安装和配置备份服务,在“块存储”节点上执行这些步骤。
博客:
https://yinwucheng.com/?p=491
https://www.cnblogs.com/tssc/p/9877026.html
https://www.cnblogs.com/wanlipenghtml/p/10721845.html
https://docs.openstack.org/mitaka/zh_CN/install-guide-rdo/cinder-storage-install.html
- 存储节点建议单独部署服务器(最好是物理机),测试时也可以部署在控制节点或者计算节点
- 在本文,存储节点使用LVM逻辑卷提供服务,需要提供一块空的磁盘用以创建LVM逻辑卷
- 这里在VMware虚拟机增加一块100GB的磁盘
- 在部署之前准备好存储设备。
一、存储节点准备好存储设备
0.磁盘挂载
在搭建的过程中使用的是VMware Workstation虚拟机来完成的,需要给存储节点新添加一个虚拟磁盘,并将磁盘挂载到虚拟机。
(1).分区
增加了空间的硬盘是 /dev/sdb
[root@controller ~]# fdisk -l
Disk /dev/sda: 68.7 GB, 68719476736 bytes, 134217728 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000a0797
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 2099199 1048576 83 Linux
/dev/sda2 2099200 134217727 66059264 8e Linux LVM
Disk /dev/sdb: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/centos-root: 42.7 GB, 42651877376 bytes, 83304448 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/centos-swap: 4160 MB, 4160749568 bytes, 8126464 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/centos-home: 20.8 GB, 20824719360 bytes, 40673280 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
[root@controller ~]#
(2).分区
[root@controller ~]# fdisk /dev/sdb
Welcome to fdisk (util-linux 2.23.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Command (m for help): p
Disk /dev/sdb: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xfa73ca8f
Device Boot Start End Blocks Id System
Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1): 3
First sector (2048-41943039, default 2048): #回车
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-41943039, default 41943039): #回车
Using default value 41943039
Partition 3 of type Linux and of size 20 GiB is set
Command (m for help): t
Selected partition 3
Hex code (type L to list all codes): w
Changed type of partition 'Linux' to 'unknown'
Command (m for help): q
[root@controller ~]# cd /dev/
[root@controller dev]# partprobe
[root@controller dev]# ls sd*
sda sda1 sda2 sdb
[root@controller dev]#
(3).格式化
[root@controller dev]# mkfs.ext3 /dev/sdb
mke2fs 1.42.9 (28-Dec-2013)
/dev/sdb is entire device, not just one partition!
Proceed anyway? (y,n) y
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
1310720 inodes, 5242880 blocks
262144 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
160 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
[root@controller dev]#
1.安装LVM相关软件包
安装LVM包
yum install lvm2 device-mapper-persistent-data -y
启动LVM的metadata服务并配置开机自启动
systemctl start lvm2-lvmetad.service
systemctl status lvm2-lvmetad.service
systemctl enable lvm2-lvmetad.service
systemctl list-unit-files |grep lvm2-lvmetad |grep enabled
2.创建LVM物理卷
创建LVM物理卷/dev/sdb
# 检查磁盘状态
[root@controller ~]# fdisk -l
# 创建LVM 物理卷 /dev/sdb
[root@controller ~]# pvcreate /dev/sdb
Physical volume "/dev/sdb" successfully created.
3.创建 LVM 卷组 cinder-volumes
(1)块存储服务会在这个卷组中创建逻辑卷
[root@controller ~]# vgcreate cinder-volumes /dev/sdb
Volume group "cinder-volumes" successfully created
[root@controller ~]# pvdisplay
--- Physical volume ---
PV Name /dev/sda2
VG Name centos
PV Size <63.00 GiB / not usable 3.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 16127
Free PE 1
Allocated PE 16126
PV UUID aTMOV5-l175-850d-vuur-0LRK-HGiz-JBLcwZ
--- Physical volume ---
PV Name /dev/sdb
VG Name cinder-volumes
PV Size 20.00 GiB / not usable 4.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 5119
Free PE 5119
Allocated PE 0
PV UUID 2GmNUZ-03a6-I2dH-hG0W-4ISc-5Liy-30XbrO
[root@controller ~]#
4.配置过滤器,只有实例可以访问块存储卷组,防止系统出错
(1)新增磁盘sdb为LVM卷组
- 默认只会有openstack实例访问块存储卷组,不过,底层的操作系统也会管理这些设备并尝试将逻辑卷与系统关联。
- 默认情况下LVM卷扫描工具会扫描整个/dev目录,查找所有包含lvm卷的块存储设备。如果其他项目在某个磁盘设备sda,sdc等上使用了lvm卷,那么扫描工具检测到这些卷时会尝试缓存这些lvm卷,可能导致底层操作系统或者其他服务无法正常调用他们的lvm卷组,从而产生各种问题,需要手动配置LVM,让LVM卷扫描工具只扫描包含"cinder-volume"卷组的设备/dev/sdb,我这边磁盘分区都是格式化的手工分区,目前不存在这个问题,以下是配置演示
将LVM重新配置为仅扫描包含cinder-volumes卷组的设备
vim /etc/lvm/lvm.conf
-----------------------------
devices {
filter = [ "a/sdb/", "r/.*/"]
}
-----------------------------
在devices部分中,添加一个接受/dev/sdb设备并拒绝所有其他设备的过滤器:
- 配置规则:
- a用于接受,r用于拒绝。
- 每个过滤器组中的元素都以a开头accept接受,或以 r 开头reject拒绝,后面连接设备名称的正则表达式规则。
- 过滤器组必须以"r/.*/"结束,过滤所有保留设备。
- 可以使用命令:vgs -vvvv来测试过滤器。
(2)、查看系统磁盘卷类型
[root@controller ~]# vgdisplay
WARNING: Device for PV aTMOV5-l175-850d-vuur-0LRK-HGiz-JBLcwZ not found or rejected by a filter.
Couldn't find device with uuid aTMOV5-l175-850d-vuur-0LRK-HGiz-JBLcwZ.
WARNING: Couldn't find all devices for LV centos/swap while checking used and assumed devices.
WARNING: Couldn't find all devices for LV centos/home while checking used and assumed devices.
WARNING: Couldn't find all devices for LV centos/root while checking used and assumed devices.
--- Volume group ---
VG Name centos
System ID
Format lvm2
Metadata Areas 0
Metadata Sequence No 4
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 3
Open LV 3
Max PV 0
Cur PV 1
Act PV 0
VG Size <63.00 GiB
PE Size 4.00 MiB
Total PE 16127
Alloc PE / Size 16126 / 62.99 GiB
Free PE / Size 1 / 4.00 MiB
VG UUID Ws4wQS-K0VZ-vp9q-TB9Y-sLP9-OKyb-SZbTd8
--- Volume group ---
VG Name cinder-volumes
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 18
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 4
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size <20.00 GiB
PE Size 4.00 MiB
Total PE 5119
Alloc PE / Size 4874 / <19.04 GiB
Free PE / Size 245 / 980.00 MiB
VG UUID AOqwgQ-yYNb-7anP-1tr4-1e3m-Bxth-b3R9rP
[root@controller ~]#
- 注意:
- 如果存储节点的操作系统磁盘/dev/sda使用的是LVM卷组,也需要将该设备添加到过滤器中,配置文件/etc/lvm/lvm.conf添加如下:
devices {
......
filter = [ "a/sda/", "a/sdb/", "r/.*/"]
......
}
- 如果计算节点的操作系统磁盘/dev/sda使用的是LVM卷组,也需要将该设备添加到过滤器中,配置文件/etc/lvm/lvm.conf添加如下:
devices {
......
filter = [ "a/sda/", "r/.*/"]
......
}
二、存储节点cinder相关软件安装与配置
1.安装相关软件
yum install openstack-cinder targetcli python-keystone -y
#用于快速配置
yum install openstack-utils -y
2.存储节点快速修改cinder配置
/etc/cinder/cinder.conf
cp /etc/cinder/cinder.conf /etc/cinder/cinder.conf.bak
grep -Ev '#|^$' /etc/cinder/cinder.conf.bak>/etc/cinder/cinder.conf
openstack-config --set /etc/cinder/cinder.conf database connection mysql+pymysql://cinder:CINDER_DBPASS@controller/cinder
openstack-config --set /etc/cinder/cinder.conf DEFAULT transport_url rabbit://openstack:RABBIT_USER_PASS@controller
openstack-config --set /etc/cinder/cinder.conf DEFAULT auth_strategy keystone
openstack-config --set /etc/cinder/cinder.conf keystone_authtoken www_authenticate_uri http://controller:5000
openstack-config --set /etc/cinder/cinder.conf keystone_authtoken auth_url http://controller:5000
openstack-config --set /etc/cinder/cinder.conf keystone_authtoken memcached_servers controller:11211
openstack-config --set /etc/cinder/cinder.conf keystone_authtoken auth_type password
openstack-config --set /etc/cinder/cinder.conf keystone_authtoken project_domain_name default
openstack-config --set /etc/cinder/cinder.conf keystone_authtoken user_domain_name default
openstack-config --set /etc/cinder/cinder.conf keystone_authtoken project_name service
openstack-config --set /etc/cinder/cinder.conf keystone_authtoken username cinder
openstack-config --set /etc/cinder/cinder.conf keystone_authtoken password CINDER_PASS
openstack-config --set /etc/cinder/cinder.conf DEFAULT my_ip 10.0.0.41
openstack-config --set /etc/cinder/cinder.conf DEFAULT enabled_backends lvm
openstack-config --set /etc/cinder/cinder.conf DEFAULT glance_api_servers http://controller:9292
openstack-config --set /etc/cinder/cinder.conf lvm volume_driver cinder.volume.drivers.lvm.LVMVolumeDriver
openstack-config --set /etc/cinder/cinder.conf lvm volume_group cinder-volumes
openstack-config --set /etc/cinder/cinder.conf lvm target_protocol iscsi
openstack-config --set /etc/cinder/cinder.conf lvm target_helper lioadm
openstack-config --set /etc/cinder/cinder.conf oslo_concurrency lock_path /var/lib/cinder/tmp
#Replace RABBIT_PASS with the password you chose for the openstack account in RabbitMQ.
#Replace MANAGEMENT_INTERFACE_IP_ADDRESS with the IP address of the management network interface on your storage node, typically 10.0.0.41 for the first node.
#注意具体配置与之前版本有所变更
#[lvm]
# volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver
# volume_group = cinder-volumes
# target_protocol = iscsi
# target_helper = lioadm
egrep -v "^#|^$" /etc/cinder/cinder.conf
grep '^[a-z]' /etc/cinder/cinder.conf
3.存储节点启动cinder服务并配置开机自启动
需要启动2个服务
systemctl start openstack-cinder-volume.service target.service
systemctl status openstack-cinder-volume.service target.service
systemctl enable openstack-cinder-volume.service target.service
systemctl list-unit-files |grep openstack-cinder |grep enabled
systemctl list-unit-files |grep target.service |grep enabled
至此存储节点的cinder服务安装完毕
至此,在存储节点安装cinder服务就完成了
三、(可选)安装和配置备份服务
#安装软件包:
yum install openstack-cinder
#编辑/etc/cinder/cinder.conf文件并完成以下操作
---------------------------------------------------------------
[DEFAULT]
# ...
backup_driver = cinder.backup.drivers.swift.SwiftBackupDriver
backup_swift_url = SWIFT_URL
---------------------------------------------------------------
#替换SWIFT_URL为对象存储服务的URL。可以通过显示对象库API端点来找到URL:
openstack catalog show object-store
#确定安装,启动块存储备份服务,并将其配置为在系统启动时启动:
systemctl start openstack-cinder-backup.service
systemctl status openstack-cinder-backup.service
systemctl enable openstack-cinder-backup.service
systemctl list-unit-files |grep openstack-cinder-backup.service |grep enabled