openstack 使用cinder nfs作为后端存储报错

1.问题：磁盘访问权限问题

报错信息：

 internal error: qemu unexpectedly closed the monitor: 2020-12-16T08:17:54.719315Z qemu-kvm: -drive file=/root/test.qcow2,format=raw,if=none,id=drive-virtio-disk0,cache=none,aio=native: Could not open '/root/test.qcow2': Permission denied

解决方法：

在计算节点上修改：

sed -i 's/^#user = \"root\"/user = \"root\"/g'  /etc/libvirt/qemu.conf
sed -i 's/^#group = \"root\"/group = \"root\"/g'  /etc/libvirt/qemu.conf

systemctl restart openstack-nova-compute

 

2.问题：非首次下发虚拟机时，重复挂载nfs文件系统报错导致虚拟机不能下发

报错信息：

Stderr: u'mount.nfs: /var/lib/nova/mnt/e1bda570eff9a173b2d193986ed9bd68 is busy or already mounted\n'
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743] Traceback (most recent call last):
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2438, in _build_and_run_instance
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]     block_device_info=block_device_info)
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3550, in spawn
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]     mdevs=mdevs)
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 6157, in _get_guest_xml
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]     context, mdevs)
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5820, in _get_guest_config
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]     flavor, guest.os_type)
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4580, in _get_guest_storage_config
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]     self._connect_volume(context, connection_info, instance)
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1650, in _connect_volume
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]     vol_driver.connect_volume(connection_info, instance)
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/volume/fs.py", line 117, in connect_volume
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]     self._mount_options(connection_info))
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/volume/mount.py", line 411, in mount
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]     options)
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/volume/mount.py", line 307, in mount
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]     nova.privsep.fs.mount(fstype, export, mountpoint, options)
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]   File "/usr/lib/python2.7/site-packages/oslo_privsep/priv_context.py", line 245, in _wrap
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]     return self.channel.remote_call(name, args, kwargs)
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]   File "/usr/lib/python2.7/site-packages/oslo_privsep/daemon.py", line 224, in remote_call
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]     raise exc_type(*result[2])
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743] ProcessExecutionError: Unexpected error while running command.
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743] Command: mount -t nfs 192.168.130.203:/data/file01 /var/lib/nova/mnt/e1bda570eff9a173b2d193986ed9bd68
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743] Exit code: 32
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743] Stdout: u''
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743] Stderr: u'mount.nfs: /var/lib/nova/mnt/e1bda570eff9a173b2d193986ed9bd68 is busy or already mounted\n'
2020-12-16 16:33:27.084 79881 ERROR nova.compute.manager [instance: 14b8c2c2-29c9-4fa3-a13b-1b3cdc57b743]

 

解决方法：

在计算节点上修改

vim /usr/lib/python2.7/site-packages/nova/virt/libvirt/volume/fs.py

修改类class LibvirtMountedFileSystemVolumeDriver(LibvirtBaseFileSystemVolumeDriver)的函数connect_volume为如下：

    def connect_volume(self, connection_info, instance):
        """Connect the volume."""
        export = connection_info['data']['export']
        vol_name = connection_info['data']['name']
        mountpoint = self._get_mount_path(connection_info)
        if not os.path.ismount(mountpoint):   #判断路径是否存在，不存在则进行挂载
           mount.mount(self.fstype, export, vol_name, mountpoint, instance, self._mount_options(connection_info))
        connection_info['data']['device_path'] = \
            self._get_device_path(connection_info)

重启服务：systemctl restart openstack-nova-compute

已经提交bug：https://bugs.launchpad.net/openstack-i18n/+bug/1908379

-------------------

改bug已经在i18n中修复了，但由于/usr/lib/systemd/system/openstack-nova-compute.service中默认是以nova权限运行，而挂载的fs卷为root:nova权限，os.path.ismount()没有root权限不能访问root:nova 目录

讲/usr/lib/systemd/system/openstack-nova-compute.service中的user设置为root即可