亲测100%可成功的openstack实例热迁移+冷迁移 openstack-Mitak（双节点）+centos7.2-1511

Description：最近实验好多次热迁移都是败了，网上的博客大多数都无法成功的进行热迁移，最后还是在老外的一片关于opensatck live migration 说明上找到的原因：[libvirtd] 没有配置  block_migration_flag= VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE,VIR_MIGRATE_NON_SHARED_INC

附上地址：http://remarks.peurat.net/rdo-live-migrationblock-migration/  （需要翻墙哦）

实验工具：

openstack-Mitaka双节点（CentOS 7.2 1511版）
host	IP
controller-node	192.168.100.10
compute-node1（控制节点上的计算服务）	192.168.100.10
compute-node2	192.168.100.20

一、首先看下热迁移，后期再写使用NFS,ceph等外部存储的实例热迁移；（本次热迁移不使用任何外部的存储，如NFS、ceph等。）：

 

1、保证两个节点的nova用户uid，gid相同，可使用id nova命令查看：

[root@compute ~]# id nova
uid=162(nova) gid=162(nova) groups=162(nova),99(nobody),107(qemu)

[root@controller ~]# id nova
uid=162(nova) gid=162(nova) groups=162(nova),99(nobody),107(qemu)


如果不相同着修改uid，gid：
# [root@compute ~]# usermod -u uid nova  XXX
# [root@compute ~]# gropumod -g gid nova XXX

2、修改libvirt配置文件（注意：两个节点都需要修改）：

[root@controller ~]# vi /etc/sysconfig/libvirtd 
# Override the default config file
# NOTE: This setting is no longer honoured if using
# systemd. Set '--config /etc/libvirt/libvirtd.conf'
# in LIBVIRTD_ARGS instead.
LIBVIRTD_CONFIG=/etc/libvirt/libvirtd.conf

# Listen for TCP/IP connections
# NB. must setup TLS/SSL keys prior to using this
LIBVIRTD_ARGS="--listen"



[root@compute ~]# vi /etc/sysconfig/libvirtd 
# Override the default config file
# NOTE: This setting is no longer honoured if using
# systemd. Set '--config /etc/libvirt/libvirtd.conf'
# in LIBVIRTD_ARGS instead.
LIBVIRTD_CONFIG=/etc/libvirt/libvirtd.conf

# Listen for TCP/IP connections
# NB. must setup TLS/SSL keys prior to using this
LIBVIRTD_ARGS="--listen"

 

[root@controller ~]# grep -vE '^#|^$' /etc/libvirt/libvirtd.conf       
listen_tls = 0
listen_tcp = 1
auth_tcp = "none"



[root@compute ~]# grep -vE '^#|^$' /etc/libvirt/libvirtd.conf       
listen_tls = 0
listen_tcp = 1
auth_tcp = "none"


两个节点都重启服务：systemctl restart libvirtd.service
两个节点都检查端口：ss -ntl | grep 16509

[root@compute ~]# ss -ntl | grep 16509
LISTEN     0      30           *:16509                    *:*    

[root@controller ~]# ss -ntl | grep 16509
LISTEN     0      30           *:16509                    *:*                  
LISTEN     0      30          :::16509                   :::*   

再来测试下可否无密连接：
[root@controller ~]# virsh -c qemu+tcp://compute/system 
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # 

[root@compute ~]# virsh -c qemu+tcp://controller/system 
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # 

3、修改nova.conf配置文件，使其允许迁移。

[root@controller ~]# vi /etc/nova/nova.conf 
[libvirt]
inject_key = True
live_migration_flag = VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE,VIR_MIGRATE_TUNNELLED
block_migration_flag = VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE,VIR_MIGRATE_NON_SHARED_INC
live_migration_downtime  =  500 
live_migration_downtime_steps  =  10 
live_migration_downtime_delay  =  75




[root@compute ~]# vi /etc/nova/nova.conf 
[libvirt]
inject_key = True
live_migration_flag = VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE,VIR_MIGRATE_TUNNELLED
block_migration_flag = VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE,VIR_MIGRATE_NON_SHARED_INC
live_migration_downtime  =  500 
live_migration_downtime_steps  =  10 
live_migration_downtime_delay  =  75


重启两个节点nova的所有服务:

在[libvirtd]选项中：一定要把 
block_migration_flag==VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE,VIR_MIGRATE_NON_SHARED_INC 
中的 VIR_MIGRATE_TUNNELLED删除。

否则报错：
[root@controller nova]# tail -f  nova-compute.log 
2019-05-15 16:24:50.692 4833 ERROR nova.virt.libvirt.driver [req-254721d5-5269-4c08-8d06-92ec37f6a306 c86d50b40c7b4b858952bdd24f11f9f9 bd8bb97a45cb4e6781c7f51952d16ffa - - -] [instance: 02e0808d-dae1-42e9-ae91-0e0a00f0f524] Live Migration failure: Operation not supported: Selecting disks to migrate is not implemented for tunnelled migration
2019-05-15 16:24:50.752 4833 ERROR nova.virt.libvirt.driver [req-254721d5-5269-4c08-8d06-92ec37f6a306 c86d50b40c7b4b858952bdd24f11f9f9 bd8bb97a45cb4e6781c7f51952d16ffa - - -] [instance: 02e0808d-dae1-42e9-ae91-0e0a00f0f524] Migration operation has aborted

4、开始迁移：

[root@controller ~]# nova show test
+--------------------------------------+----------------------------------------------------------+
| Property                             | Value                                                    |
+--------------------------------------+----------------------------------------------------------+
| OS-DCF:diskConfig                    | AUTO                                                     |
| OS-EXT-AZ:availability_zone          | nova                                                     |
| OS-EXT-SRV-ATTR:host                 | controller                                               |
| OS-EXT-SRV-ATTR:hostname             | test                                                    |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | controller                                               |
| OS-EXT-SRV-ATTR:instance_name        | instance-0000001c                                        |



[root@controller ~]# nova live-migration 02e0808d-dae1-42e9-ae91-0e0a00f0f524  compute
[root@controller ~]# nova show test
+--------------------------------------+----------------------------------------------------------+
| Property                             | Value                                                    |
+--------------------------------------+----------------------------------------------------------+
| OS-DCF:diskConfig                    | AUTO                                                     |
| OS-EXT-AZ:availability_zone          | nova                                                     |
| OS-EXT-SRV-ATTR:host                 | compute                                                  |
| OS-EXT-SRV-ATTR:hostname             | test                                                     |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | compute                                                  |
| OS-EXT-SRV-ATTR:instance_name        | instance-0000001c                                        |
| OS-EXT-SRV-ATTR:kernel_id            |                                                          |
| OS-EXT-SRV-ATTR:launch_index         | 0                                                        |
| OS-EXT-SRV-ATTR:ramdisk_id           |                                                          |
| OS-EXT-SRV-ATTR:reservation_id       | r-x3iil7sr                                               |
| OS-EXT-SRV-ATTR:root_device_name     | /dev/vda                                                 |
| OS-EXT-SRV-ATTR:user_data            | -                                                        |
| OS-EXT-STS:power_state               | 1                                                        |
| OS-EXT-STS:task_state                | -                                                        |
| OS-EXT-STS:vm_state                  | active                                                   |
| OS-SRV-USG:launched_at               | 2019-05-15T18:58:31.000000                               |
| OS-SRV-USG:terminated_at             | -                                                        |
| accessIPv4                           |                                                          |
| accessIPv6                           |                                                          |
| config_drive                         |                                                          |
| created                              | 2019-05-15T18:58:19Z                                     |
| description                          | test                                                     |
| flavor                               | m1.small (2)                                             |
| hostId                               | 80b6b07ccdf4b55333337f3f2d1e1b687a97301580c11a27806327ac |
| host_status                          | UP                                                       |
| id                                   | 02e0808d-dae1-42e9-ae91-0e0a00f0f524                     |
| image                                | centos-7.2 (6c3b67e7-d765-4a8a-8d26-97a5fd52971e)        |
| init-net1 network                    | 10.0.0.44                                                |
| key_name                             | -                                                        |
| locked                               | False                                                    |
| metadata                             | {}                                                       |
| name                                 | test                                                    |
| os-extended-volumes:volumes_attached | []                                                       |
| progress                             | 0                                                        |
| security_groups                      | default                                                  |
| status                               | ACTIVE                                                   |
| tenant_id                            | bd8bb97a45cb4e6781c7f51952d16ffa                         |
| updated                              | 2019-05-15T19:48:35Z                                     |
| user_id                              | c86d50b40c7b4b858952bdd24f11f9f9                         |
+--------------------------------------+----------------------------------------------------------+

到此热迁移完成；

二、冷迁移--冷迁移相对于热迁移要简单很多：

流程：将某计算节点的实例关机——>将/var/lib/nova/instances/下对应的实例使用scp命令配合  -r  参数传输到目的主机的/var/lib/nova/instances/——>使用chown命令修改文件所有者——>手动修改nova数据的的instances表的host与node字段——>将虚拟机开机——>冷迁移完成

[root@controller ~]# nova show test
+--------------------------------------+----------------------------------------------------------+
| Property                             | Value                                                    |
+--------------------------------------+----------------------------------------------------------+
| OS-DCF:diskConfig                    | AUTO                                                     |
| OS-EXT-AZ:availability_zone          | nova                                                     |
| OS-EXT-SRV-ATTR:host                 | controller                                               |
| OS-EXT-SRV-ATTR:hostname             | test                                                     |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | controller                                               |

看清楚这台实例实在controller上，

[root@controller ~]#nova stop ceph

[root@controller ~]# scp -r  /var/lib/nova/instances/02e0808d-dae1-42e9-ae91-0e0a00f0f524/ compute:/var/lib/nova/instances/

[root@compute ~]#chown nova:nova /var/lib/nova/instances/02e0808d-dae1-42e9-ae91-0e0a00f0f524/
如果还需要其他权限使用 ：chmod +777 -R /var/lib/nova/instances/02e0808d-dae1-42e9-ae91-0e0a00f0f524/

修改数据库：
MariaDB [(none)]> update nova.instances set host='compute',node='compute' where uuid='02e0808d-dae1-42e9-ae91-0e0a00f0f524';

重启下服务：

[root@compute ~]#systemctl restart openstack-nova-compute

开启实例：

[root@controller ~]#nova start ceph


[root@controller ~]# nova show test
+--------------------------------------+----------------------------------------------------------+
| Property                             | Value                                                    |
+--------------------------------------+----------------------------------------------------------+
| OS-DCF:diskConfig                    | AUTO                                                     |
| OS-EXT-AZ:availability_zone          | nova                                                     |
| OS-EXT-SRV-ATTR:host                 | compute                                            |
| OS-EXT-SRV-ATTR:hostname             | test                                                   |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | compute