正常情况下删除云主机的操作,主要采用如下两个方式:
1. is_local_delete = True 采用local_delete()
2. is_local_delete = False 采用compute_rpcapi.terminate_instance()
云主机在如下的状态下vm_statesvm_states.SHELVED, vm_states.SHELVED_OFFLOADED下,会采用其他方式。
当发生如下两种常见场景时,会采用上述的删除流程:
1)nova-compute已经down了一段时间,nova_api检测服务状态时(nova service-list),发现nova-compute已经down(但
该节点的所有云主机进程还在)
2)nova api 检测服务状态时(nova service-list),发现nova-compute正常(但nova_compute服务可能正常,也可能down)。
检测方式:nova-compute每隔10s向数据库写入一次更新时间,nova-api检测时会根据该更新时间来作判断。当超过service_down_time=60s,则会认为nova-compute服务已经down
两种删除场景,会出现如下问题(后端都采用ceph存储):
第一种场景:当云主机是从cinder volume创建时,删除该云主机会存在系统盘残留。
原因:系统盘对应的rbd image被加锁
第二种场景:当nova_api检测时,发现compute是正常的,但是实际已经down了。
此时,删除云主机,会一直处于deleting状态,直达nova_compute正常时,才会正常删除
注释:ceph创建rbd时默认是不加锁的,只有当被使用时,才会对其加锁。第一种场景时,系统盘创建后会被attach到云主机,因此会被加锁。
正常情况下,硬盘挂载到云主机时,是不会加锁的,直到在云主机内将其mount
代码重点分析(liberty):
nova/compute/api.py
def _delete_instance(self, context, instance):
self._delete(context, instance, 'delete', self._do_delete,
task_state=task_states.DELETING)
#####cb=self._do_delete(),instance_attrs= task_states.DELETING
def _delete(self, context, instance, delete_type, cb, **instance_attrs):
if instance.disable_terminate:
LOG.info(_LI('instance termination disabled'),
instance=instance)
return
bdms = objects.BlockDeviceMappingList.get_by_instance_uuid(
context, instance.uuid)
project_id, user_id = quotas_obj.ids_from_instance(context, instance)
# At these states an instance has a snapshot associate.
if instance.vm_state in (vm_states.SHELVED,
vm_states.SHELVED_OFFLOADED):
snapshot_id = instance.system_metadata.get('shelved_image_id')
LOG.info(_LI("Working on deleting snapshot %s "
"from shelved instance..."),
snapshot_id, instance=instance)
try:
self.image_api.delete(context, snapshot_id)
except (exception.ImageNotFound,
exception.ImageNotAuthorized) as exc:
LOG.warning(_LW("Failed to delete snapshot "