openstack_F_scheduler虚拟机创建调度

本文地址：http://blog.csdn.net/leoduo2013/article/details/16826419

创建一个实例，经历的调度过程主要有以下几个过程：

1.      nova.scheduler.manager.py::SchedulerManager

模块收到RPC消息，调用run_instance()函数：

    def run_instance(self, context, request_spec, admin_password,
            injected_files, requested_networks, is_first_time,
            filter_properties):
        """Tries to call schedule_run_instance on the driver.
        Sets instance vm_state to ERROR on exceptions
        """
        try:
            return self.driver.schedule_run_instance(context,
                    request_spec, admin_password, injected_files,
                    requested_networks, is_first_time, filter_properties)
        except exception.NoValidHost as ex:
            # don't re-raise
            self._set_vm_state_and_notify('run_instance',
                                         {'vm_state': vm_states.ERROR,
                                          'task_state': None},
                                          context, ex, request_spec)
        except Exception as ex:
            with excutils.save_and_reraise_exception():
                self._set_vm_state_and_notify('run_instance',
                                             {'vm_state': vm_states.ERROR,
                                              'task_state': None},
                                             context, ex, request_spec)

如上run_instance()方法中调用driver对象schedule_run_instance()方法。

SchedulerManager类的初始化函数__init__():

    def __init__(self, scheduler_driver=None, *args, **kwargs):
        if not scheduler_driver:
            scheduler_driver = FLAGS.scheduler_driver
        self.driver = importutils.import_object(scheduler_driver)
        super(SchedulerManager, self).__init__(*args, **kwargs)

在SchedulerManager.py最开始部分可以找到：

scheduler_driver_opt = cfg.StrOpt('scheduler_driver',
        default='nova.scheduler.multi.MultiScheduler',
        help='Default driver to use for the scheduler')

scheduler_driver字段表示的类对象。通过配置文件看到这里默认是nova.scheduler.muti.MultiScheduler类。

2.       MultiScheduler::schedule_run_instance()

    def schedule_run_instance(self, *args, **kwargs):
        return self.drivers['compute'].schedule_run_instance(*args, **kwargs)

在MultiScheduler类的初始化函数中找到drivers字典的定义，发现其“compute”键对应的值是nova.scheduler.filter_scheduler.FilterScheduler类对象。

也就是说创建虚拟机用的是FilterScheduler而不再是chance.py中的ChanceScheduler

3.      FilterScheduler::schedule_run_instance()源码如下：

    def schedule_run_instance(self, context, request_spec,
                              admin_password, injected_files,
                              requested_networks, is_first_time,
                              filter_properties):
        """This method is called from nova.compute.api to provision
        an instance.  We first create a build plan (a list of WeightedHosts)
        and then provision.

        Returns a list of the instances created.
        """
        elevated = context.elevated()
        instance_uuids = request_spec.get('instance_uuids')
        num_instances = len(instance_uuids)
        LOG.debug(_("Attempting to build %(num_instances)d instance(s)") %
                locals())

        payload = dict(request_spec=request_spec)
        notifier.notify(context, notifier.publisher_id("scheduler"),
                        'scheduler.run_instance.start', notifier.INFO, payload)

        # a)"filter,weight then update hosts' resources and return hosts needed"
        weighted_hosts = self._schedule(context, "compute", request_spec,
                                        filter_properties, instance_uuids)

        # NOTE(comstud): Make sure we do not pass this through.  It
        # contains an instance of RpcContext that cannot be serialized.
        filter_properties.pop('context', None)

        for num, instance_uuid in enumerate(instance_uuids):
            request_spec['instance_properties']['launch_index'] = num

            try:
                try:
                    weighted_host = weighted_hosts.pop(0)
                except IndexError:
                    raise exception.NoValidHost(reason="")

                # b)"create instances in hosts listed in weighted_host"
                self._provision_resource(elevated, weighted_host,
                                         request_spec,
                                         filter_properties,
                                         requested_networks,
                                         injected_files, admin_password,
                                         is_first_time,
                                         instance_uuid=instance_uuid)
            except Exception as ex:
                # NOTE(vish): we don't reraise the exception here to make sure
                #             that all instances in the request get set to
                #             error properly
                driver.handle_schedule_error(context, ex, instance_uuid,
                                             request_spec)
            # scrub retry host list in case we're scheduling multiple
            # instances:
            retry = filter_properties.get('retry', {})
            retry['hosts'] = []
        # 循环结束后，返回虚拟机信息，调用链依次返回，给用户响应
        notifier.notify(context, notifier.publisher_id("scheduler"),
                        'scheduler.run_instance.end', notifier.INFO, payload)

该方法主要做了两件事：

 a)     首先调用了_schedule()方法。

    def _schedule(self, context, topic, request_spec, filter_properties,
                  instance_uuids=None):
        """Returns a list of hosts that meet the required specs,
        ordered by their fitness.
        """
        elevated = context.elevated()
        if topic != "compute":
            msg = _("Scheduler only understands Compute nodes (for now)")
            raise NotImplementedError(msg)

        instance_properties = request_spec['instance_properties']
        instance_type = request_spec.get("instance_type", None)

        cost_functions = self.get_cost_functions()
        config_options = self._get_configuration_options()

        # check retry policy.  Rather ugly use of instance_uuids[0]...
        # but if we've exceeded max retries... then we really only
        # have a single instance.
        properties = instance_properties.copy()
        if instance_uuids:
            properties['uuid'] = instance_uuids[0]
        self._populate_retry(filter_properties, properties)

        filter_properties.update({'context': context,
                                  'request_spec': request_spec,
                                  'config_options': config_options,
                                  'instance_type': instance_type})

        self.populate_filter_properties(request_spec,
                                        filter_properties)

        # Find our local list of acceptable hosts by repeatedly
        # filtering and weighing our options. Each time we choose a
        # host, we virtually consume resources on it so subsequent
        # selections can adjust accordingly.

        # unfiltered_hosts_dict is {host : ZoneManager.HostInfo()}
        # 获取所有的计算节点主机
        unfiltered_hosts_dict = self.host_manager.get_all_host_states(
                elevated, topic)

        # Note: remember, we are using an iterator here. So only
        # traverse this list once. This can bite you if the hosts
        # are being scanned in a filter or weighing function.
        hosts = unfiltered_hosts_dict.itervalues()

        selected_hosts = []
        if instance_uuids:
            num_instances = len(instance_uuids)
        else:
            num_instances = request_spec.get('num_instances', 1)
        for num in xrange(num_instances):
            # Filter local hosts based on requirements ...
            # i.对主机进行过滤
            hosts = self.host_manager.filter_hosts(hosts,
                    filter_properties)
            if not hosts:
                # Can't get any more locally.
                break

            LOG.debug(_("Filtered %(hosts)s") % locals())

            # weighted_host = WeightedHost() ... the best
            # host for the job.
            # TODO(comstud): filter_properties will also be used for
            # weighing and I plan fold weighing into the host manager
            # in a future patch.  I'll address the naming of this
            # variable at that time.
            # ii.对所有通过过滤的主机计算权值,选取权值最小主机
            weighted_host = least_cost.weighted_sum(cost_functions,
                    hosts, filter_properties)
            LOG.debug(_("Weighted %(weighted_host)s") % locals())
            selected_hosts.append(weighted_host)

            # Now consume the resources so the filter/weights
            # will change for the next instance.
            # iii.对选择的主机资源进行更新，以便在下一次循环中使用最新的数据计算。
            weighted_host.host_state.consume_from_instance(
                    instance_properties)

        selected_hosts.sort(key=operator.attrgetter('weight'))
        return selected_hosts

父类中self.host_manager = importutils.import_object(FLAGS.scheduler_host_manager )

scheduler_host_manager默认字段为nova.scheduler.host_manager.HostManager，即应用的该类。

 获取所有的计算节点主机方法 self.host_manager.get_all_host_states(elevated, topic)和 对主机进行过滤方法 self.host_manager.filter_hosts(hosts, filter_properties)都在该类中.

i.对主机进行过滤,代码如下：

    def filter_hosts(self, hosts, filter_properties, filters=None):
        """Filter hosts and return only ones passing all filters"""
        filtered_hosts = []
        # (1)返回所有可用filter的host_passes方法：
        filter_fns = self._choose_host_filters(filters)
        # (2)依次对每个host调用filter类的host_passes方法，如果返回都为True，则主机通过过滤。
        for host in hosts:
            if host.passes_filters(filter_fns, filter_properties):
                filtered_hosts.append(host)
        return filtered_hosts

FilterManager继承自Scheduler类。在Scheduler的初始化中，加载了所有可用的filter类。根据配置文件中scheduler_default_filters字段的定义选择默认使用的一个或多个filter。

（1）_choose_host_filters()返回所有可用filter的host_passes方法：

    def _choose_host_filters(self, filters):
        """Since the caller may specify which filters to use we need
        to have an authoritative list of what is permissible. This
        function checks the filter names against a predefined set
        of acceptable filters.
        """
        if filters is None:
            filters = FLAGS.scheduler_default_filters
        if not isinstance(filters, (list, tuple)):
            filters = [filters]
        good_filters = []
        bad_filters = []
        for filter_name in filters:
            found_class = False
            for cls in self.filter_classes:
                if cls.__name__ == filter_name:
                    found_class = True
                    filter_instance = cls()
                    #####################################
                    # Get the filter function
                    filter_func = getattr(filter_instance,
                            'host_passes', None)
                    if filter_func:
                        good_filters.append(filter_func)
                    break
                    ########################################
            if not found_class:
                bad_filters.append(filter_name)
        if bad_filters:
            msg = ", ".join(bad_filters)
            raise exception.SchedulerHostFilterNotFound(filter_name=msg)
        return good_filters

（2）passes_filters()

    def passes_filters(self, filter_fns, filter_properties):
        """Return whether or not this host passes filters."""

        if self.host in filter_properties.get('ignore_hosts', []):
            LOG.debug(_('Host filter fails for ignored host %(host)s'),
                      {'host': self.host})
            return False

        force_hosts = filter_properties.get('force_hosts', [])
        if force_hosts:
            if not self.host in force_hosts:
                LOG.debug(_('Host filter fails for non-forced host %(host)s'),
                          {'host': self.host})
            return self.host in force_hosts
        # 依次对每个host调用filter类的host_passes方法，如果返回都为True，则主机通过过滤。
        for filter_fn in filter_fns:
            if not filter_fn(self, filter_properties):
                LOG.debug(_('Host filter function %(func)s failed for '
                            '%(host)s'),
                          {'func': repr(filter_fn),
                           'host': self.host})
                return False

ii.对所有通过过滤的主机计算权值。

nova.scheduler import least_cost.py::weighted_sum():

def weighted_sum(weighted_fns, host_states, weighing_properties):
    """Use the weighted-sum method to compute a score for an array of objects.

    Normalize the results of the objective-functions so that the weights are
    meaningful regardless of objective-function's range.

    :param host_list:    ``[(host, HostInfo()), ...]``
    :param weighted_fns: list of weights and functions like::

        [(weight, objective-functions), ...]

    :param weighing_properties: an arbitrary dict of values that can
        influence weights.

    :returns: a single WeightedHost object which represents the best
              candidate.
    """

    min_score, best_host = None, None
    for host_state in host_states:
        score = sum(weight * fn(host_state, weighing_properties)
                    for weight, fn in weighted_fns)
        if min_score is None or score < min_score:
            min_score, best_host = score, host_state

    return WeightedHost(min_score, host_state=best_host)

计算比较权值和，返回最小权值和对应host

openstack默认是算法是-1*(主机剩余的内存值)，选择权值最小的主机，即：选择剩余内存最大的主机创建虚拟机。（这里的策略即负载均衡，如果把默认的weight值改为1，策略就调整为优先选择一个主机创建，直至该主机内存不足）

iii.对选择的主机资源进行更新，以便在下一次循环中使用最新的数据计算。

nova.scheduler.host_manager.HostManager::consume_from_instance()

def consume_from_instance(self, instance):
        """Incrementally update host state from an instance"""
        disk_mb = (instance['root_gb'] + instance['ephemeral_gb']) * 1024
        ram_mb = instance['memory_mb']
        vcpus = instance['vcpus']
        self.free_ram_mb -= ram_mb
        self.free_disk_mb -= disk_mb
        self.vcpus_used += vcpus
        self.updated = timeutils.utcnow()

最终，_schedule()方法会得到与创建虚拟机个数一致的主机列表。

（只得到列表，更新了状态，但是还没有创建虚拟机实例）

b)     对每一个要创建的虚拟机调用：_provision_resource()：

    def _provision_resource(self, context, weighted_host, request_spec,
            filter_properties, requested_networks, injected_files,
            admin_password, is_first_time, instance_uuid=None):
        """Create the requested resource in this Zone."""
        # Add a retry entry for the selected compute host:
        self._add_retry_host(filter_properties, weighted_host.host_state.host)

        self._add_oversubscription_policy(filter_properties,
                weighted_host.host_state)

        payload = dict(request_spec=request_spec,
                       weighted_host=weighted_host.to_dict(),
                       instance_id=instance_uuid)
        notifier.notify(context, notifier.publisher_id("scheduler"),
                        'scheduler.run_instance.scheduled', notifier.INFO,
                        payload)
        # i.向数据库中添加虚拟机信息
        updated_instance = driver.instance_update_db(context, instance_uuid)
        # ii.向对应的compute主机发送创建虚拟机的异步RPC请求
        self.compute_rpcapi.run_instance(context, instance=updated_instance,
                host=weighted_host.host_state.host,
                request_spec=request_spec, filter_properties=filter_properties,
                requested_networks=requested_networks,
                injected_files=injected_files,
                admin_password=admin_password, is_first_time=is_first_time)

i.向数据库中添加虚拟机信息

def instance_update_db(context, instance_uuid):
    '''Clear the host and set the scheduled_at field of an Instance.

    :returns: An Instance with the updated fields set properly.
    '''
    now = timeutils.utcnow()
    values = {'host': None, 'scheduled_at': now}
    return db.instance_update(context, instance_uuid, values)

ii.向对应的compute主机发送创建虚拟机的异步RPC请求 nova.compute.rpcapi.py中ComputeAPI::run_instance()

def run_instance(self, ctxt, instance, host, request_spec,
                     filter_properties, requested_networks,
                     injected_files, admin_password,
                     is_first_time):
        instance_p = jsonutils.to_primitive(instance)
        self.cast(ctxt, self.make_msg('run_instance', instance=instance_p,
                request_spec=request_spec, filter_properties=filter_properties,
                requested_networks=requested_networks,
                injected_files=injected_files, admin_password=admin_password,
                is_first_time=is_first_time),
                topic=_compute_topic(self.topic, ctxt, host, None))

循环结束后，返回虚拟机信息，调用链依次返回，给用户响应FilterScheduler::schedule_run_instance()中部分代码：

模块收