场景：计算节点挂掉了，该计算节点上面的vm状态为error，并且该vm boot from image not create a new volume
1、首先记录该vm的基本信息
[root@cloud nova]# openstack server show N-server
+-----------------------------+---------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------------+---------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | AUTO |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | None |
| OS-EXT-STS:vm_state | error |
| OS-SRV-USG:launched_at | 2017-10-17T07:30:51.000000 |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| config_drive | |
| created | 2017-10-17T07:30:44Z |
| fault | {u'message': u'Unavailable console type serial.', u'code': 400, u'created': |
| | u'2017-11-09T07:12:05Z'} |
| flavor | m1.large (14) |
| hostId | 8a1d711b033c6621a00af0311397a63be484b63f1f5e0a7a3cc3dffb |
| id | 1d6a1c2e-e7d2-4320-8ee0-02ba08622fea |
| image | 916293f4-3a2d-433e-8c08-a4dad22a8d06 |
| key_name | default |
| name | N-server |
| project_id | e56ae878b9a043e7a4d91b775e4044eb |
| properties | |
| security_groups | name='default' |
| status | ERROR |
| updated | 2017-11-09T06:35:26Z |
| user_id | 4b74c59c24f94856988f08a7a240c69e |
| volumes_attached | id='5ff4b474-0af8-42cd-b9cf-c2d658b7d532' |
+-----------------------------+---------------------------------------------------------------------------------------+
2、查看/修改delete_on_termination值
由于是boot from image，所以先查看该instance所挂的volume是否属于删除instance就删除volume的类型，为了保护数据盘，进入nova数据库查看delete_on_terminatio参数的值，如果为1则改为0（1:表示删除instance时删除volume）：
[root@cloud ~]# docker exec -it -u nova mariadb mysql -h 10.0.1.2 -P 3306 -p
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 4345369
Server version: 10.0.30-MariaDB-wsrep MariaDB Server, wsrep_25.19.rc3fc46e
Copyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]>
MariaDB [(none)]> use nova;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
MariaDB [nova]> select * from block_device_mapping where instance_uuid="1d6a1c2e-e7d2-4320-8ee0-02ba08622fea";
[no+---------------------+---------------------+---------------------+------+-------------+-----------------------+-------------+--------------------------------------+-------------+-----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+---------+-------------+------------------+--------------+-------------+----------+------------+--------------------------------------+------+
| created_at | updated_at | deleted_at | id | device_name | delete_on_termination | snapshot_id | volume_id | volume_size | no_device | connection_info | instance_uuid | deleted | source_type | destination_type | guest_format | device_type | disk_bus | boot_index | image_id | tag |
+---------------------+---------------------+---------------------+------+-------------+-----------------------+-------------+--------------------------------------+-------------+-----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+---------+-------------+------------------+--------------+-------------+----------+------------+--------------------------------------+------+
| 2017-10-17 07:30:44 | 2017-10-17 07:30:45 | NULL | 457 | /dev/vda | 1 | NULL | NULL | NULL | 0 | NULL | 1d6a1c2e-e7d2-4320-8ee0-02ba08622fea | 0 | image | local | NULL | disk | NULL | 0 | 916293f4-3a2d-433e-8c08-a4dad22a8d06 | NULL |
| 2017-10-17 07:32:11 | 2017-10-24 02:03:21 | 2017-10-24 02:05:10 | 463 | /dev/vdb | 0 | NULL | 5ff4b474-0af8-42cd-b9cf-c2d658b7d532 | 200 | NULL | {"driver_volume_type": "iscsi", "connector": {"initiator": "iqn.1994-05.com.redhat:931ade18c9f9", "ip": "10.0.1.24", "platform": "x86_64", "host": "cloud", "do_local_attach": false, "os_type": "linux2", "multipath": true}, "serial": "5ff4b474-0af8-42cd-b9cf-c2d658b7d532", "data": {"device_path": "/dev/disk/by-id/dm-uuid-mpath-3600a09803830434972244a78424a706c", "target_discovered": false, "encrypted": false, "qos_specs": null, "target_iqn": "iqn.1992-08.com.netapp:sn.e10c2fc336cc11e7853000a098b226ab:vs.5", "target_portal": "10.5.2.1:3260", "volume_id": "5ff4b474-0af8-42cd-b9cf-c2d658b7d532", "target_lun": 6, "access_mode": "rw"}} | 1d6a1c2e-e7d2-4320-8ee0-02ba08622fea | 463 | volume | volume | NULL | disk | virtio | NULL | NULL | NULL |
| 2017-10-24 02:12:49 | 2017-11-08 02:11:43 | NULL | 1255 | /dev/vdb | 0 | NULL | 5ff4b474-0af8-42cd-b9cf-c2d658b7d532 | 200 | NULL | {"driver_volume_type": "iscsi", "connector": {"initiator": "iqn.1994-05.com.redhat:931ade18c9f9", "ip": "10.0.1.24", "platform": "x86_64", "host": "cloud", "do_local_attach": false, "os_type": "linux2", "multipath": true}, "serial": "5ff4b474-0af8-42cd-b9cf-c2d658b7d532", "data": {"device_path": "/dev/disk/by-id/dm-uuid-mpath-3600a09803830434972244a78424a7178", "target_discovered": false, "encrypted": false, "qos_specs": null, "target_iqn": "iqn.1992-08.com.netapp:sn.4eeb66d3b3d011e7853000a098b226ab:vs.7", "target_portal": "10.5.2.1:3260", "volume_id": "5ff4b474-0af8-42cd-b9cf-c2d658b7d532", "target_lun": 3, "access_mode": "rw"}} | 1d6a1c2e-e7d2-4320-8ee0-02ba08622fea | 0 | volume | volume | NULL | disk | virtio | NULL | NULL | NULL |
+---------------------+---------------------+---------------------+------+-------------+-----------------------+-------------+--------------------------------------+-------------+-----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+---------+-------------+------------------+--------------+-------------+----------+------------+--------------------------------------+------+
3 rows in set (0.00 sec)
MariaDB [nova]> update block_device_mapping set delete_on_termination=0 where instance_uuid="1d6a1c2e-e7d2-4320-8ee0-02ba08622fea";
Query OK, 3 rows affected (0.00 sec)
Rows matched: 3 Changed: 3 Warnings: 0
3、利用计算节点上面该vm的系统盘文件制作新镜像
[root@cloud ~]# docker volume inspect nova_compute
[
{
"Name": "nova_compute",
"Driver": "local",
"Mountpoint": "/var/lib/docker/volumes/nova_compute/_data",
"Labels": null,
"Scope": "local"
}
]
[root@cloud ~]# cd /var/lib/docker/volumes/nova_compute/_data
[root@cloud _data]# ls
buckets instances keys networks tmp
[root@cloud _data]# cd instances/
[root@cloud instances]# cd 1d6a1c2e-e7d2-4320-8ee0-02ba08622fea/
[root@cloud 1d6a1c2e-e7d2-4320-8ee0-02ba08622fea]# ls
console.log disk disk.info
[root@cloud 1d6a1c2e-e7d2-4320-8ee0-02ba08622fea]# cat disk.info
{"/var/lib/nova/instances/1d6a1c2e-e7d2-4320-8ee0-02ba08622fea/disk": "qcow2"}
[root@cloud 1d6a1c2e-e7d2-4320-8ee0-02ba08622fea]# qemu-img info disk
file format: qcow2
virtual size: 8.0G (8589934592 bytes)
disk size: 6.3G
cluster_size: 65536
backing file: /var/lib/nova/instances/_base/2b612487a6006bd6792c75e88b986aad9f43568e
Format specific information:
compat: 1.1
lazy refcounts: false
[root@cloud 1d6a1c2e-e7d2-4320-8ee0-02ba08622fea]# cp ../_base/2b612487a6006bd6792c75e88b986aad9f43568e new.qcow2 ----->生成后端镜像文件new.qcow2
[root@cloud 1d6a1c2e-e7d2-4320-8ee0-02ba08622fea]# ls
console.log disk disk.info new.qcow2
[root@cloud 1d6a1c2e-e7d2-4320-8ee0-02ba08622fea]# qemu-img rebase -b new.qcow2 disk ------->更新后端镜像文件disk
[root@cloud 1d6a1c2e-e7d2-4320-8ee0-02ba08622fea]# qemu-img commit disk
Image committed.
4、创建vm
上传new.qcow2镜像，这里使用boot from volume方式(这种方式比较安全，数据都存在共享存储上)，删除原有的instance，利用原有ip创建新的vm，先利用原有ip创建port：
[root@cloud ~]# openstack port create --fixed-ip subnet=65c1a4d8-0b1f-4d64-9b65-6077f4d251c4,ip-address=10.0.1.3 --network bf358074-4e27-4b97-8122-c64d8a3ac665 portn
+-----------------------+-----------------------------------------------------------------------------+
| Field | Value |
+-----------------------+-----------------------------------------------------------------------------+
| admin_state_up | UP |
| allowed_address_pairs | |
| binding_host_id | None |
| binding_profile | None |
| binding_vif_details | None |
| binding_vif_type | None |
| binding_vnic_type | normal |
| created_at | 2017-11-09T08:36:16Z |
| description | |
| device_id | |
| device_owner | |
| dns_assignment | None |
| dns_name | None |
| extra_dhcp_opts | |
| fixed_ips | ip_address='10.0.1.3', subnet_id='65c1a4d8-0b1f-4d64-9b65-6077f4d251c4' |
| id | e67c5c84-15cd-4b97-9ba7-5570fef159f2 |
| ip_address | None |
| mac_address | fa:16:3e:a6:13:5c |
| name | portn |
| network_id | bf358074-4e27-4b97-8122-c64d8a3ac665 |
| option_name | None |
| option_value | None |
| port_security_enabled | False |
| project_id | e56ae878b9a043e7a4d91b775e4044eb |
| qos_policy_id | None |
| revision_number | 4 |
| security_groups | aff61d17-5abd-4aca-8fed-1af03c7f83b4 |
| status | DOWN |
| subnet_id | None |
| updated_at | 2017-11-09T08:36:16Z |
+-----------------------+-----------------------------------------------------------------------------+
[root@cloud ~]# openstack server create --volume 6f881480-e937-4a16-83b5-7f9c85909944 --nic port-id=e67c5c84-15cd-4b97-9ba7-5570fef159f2 --flavor m1.large --key-name default N-server
+-----------------------------+--------------------------------------+
| Field | Value |
+-----------------------------+--------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | u9mxNz7Hqvs8 |
| config_drive | |
| created | 2017-11-09T08:38:38Z |
| flavor | m1.large (14) |
| hostId | |
| id | 402497ed-e72b-4107-bbec-0be753263faf |
| image | |
| key_name | default |
| name | N-server |
| progress | 0 |
| project_id | e56ae878b9a043e7a4d91b775e4044eb |
| properties | |
| security_groups | name='default' |
| status | BUILD |
| updated | 2017-11-09T08:38:38Z |
| user_id | 4b74c59c24f94856988f08a7a240c69e |
| volumes_attached | |
+-----------------------------+--------------------------------------+
这样就无损的恢复了error的vm，可以放心的去定位挂掉的计算节点了。
1万+
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
