cinder服务启动源码分析

感谢朋友支持本博客，欢迎共同探讨交流，由于能力和时间有限，错误之处在所难免，欢迎指正！
如果转载，请保留作者信息。
博客地址：http://blog.csdn.net/gaoxingnengjisuan
邮箱地址：dong.liu@siat.ac.cn

我们知道在/cinder/bin目录下的文件是各个服务的启动脚本，这里我们以cinder-all为例，来解析cinder服务启动流程。

首先来看代码/cinder/bin/cinder-all：

import eventlet

eventlet.monkey_patch()

import os
import sys

from oslo.config import cfg

# 路径的标准化，消除双斜线等等；
possible_topdir = os.path.normpath(os.path.join(os.path.abspath(sys.argv[0]),
                                   os.pardir,
                                   os.pardir))
if os.path.exists(os.path.join(possible_topdir, "cinder", "__init__.py")):
    sys.path.insert(0, possible_topdir)

from cinder.openstack.common import gettextutils
gettextutils.install('cinder', lazy=False)

from cinder.common import config  # Need to register global_opts
from cinder.openstack.common import log as logging
from cinder import service
from cinder import utils
from cinder import version


CONF = cfg.CONF


if __name__ == '__main__':
    CONF(sys.argv[1:], project='cinder',
         version=version.version_string())
    # 日志相关；
    logging.setup("cinder")
    LOG = logging.getLogger('cinder.all')

    utils.monkey_patch()
    servers = []
    # cinder-api
    try:
        servers.append(service.WSGIService('osapi_volume'))
    except (Exception, SystemExit):
        LOG.exception(_('Failed to load osapi_volume'))

    for binary in ['cinder-volume', 'cinder-scheduler']:
        try:
            servers.append(service.Service.create(binary=binary))
        except (Exception, SystemExit):
            LOG.exception(_('Failed to load %s'), binary)
    service.serve(*servers)
    service.wait()

在代码中我们看到 分别为'osapi_volume'，'cinder-volume'和'cinder-scheduler'初始化了三个服务类的对象，分别是cinder.service.WSGIService object，cinder.service.Service object和cinder.service.Service object，我们先来看看类cinder.service.WSGIService的对象初始化方法：

class WSGIService(object):
    """
    Provides ability to launch API from a 'paste' configuration.
    实现启动API的类；
    """

    def __init__(self, name, loader=None):
        """
        Initialize, but do not start the WSGI server.
        类的初始化方法，但是不启动服务；

        :param name: The name of the WSGI server given to the loader.
        给定的要加载的服务的名称；
        :param loader: Loads the WSGI application using the given name.
        :returns: None

        """
        # 给定的要加载的服务的名称；
        # osapi_volume
        self.name = name
        # 根据给定的服务名称导入对应的管理类；
        self.manager = self._get_manager()
        self.loader = loader or wsgi.Loader()
        self.app = self.loader.load_app(name)
        # 获取主机host；
        self.host = getattr(CONF, '%s_listen' % name, "0.0.0.0")
        # 获取主机端口；
        self.port = getattr(CONF, '%s_listen_port' % name, 0)
        # Server：管理WSGI服务的类；
        self.server = wsgi.Server(name,
                                  self.app,
                                  host=self.host,
                                  port=self.port)

可见这里进一步进行了wsgi.Server类的对象初始化操作，来看代码：

class Server(object):
    """
    Server class to manage a WSGI server, serving a WSGI application.
    管理WSGI服务的类；
    """

    default_pool_size = 1000

    def __init__(self, name, app, host=None, port=None, pool_size=None,
                 protocol=eventlet.wsgi.HttpProtocol):
        """
        Initialize, but do not start, a WSGI server.
        类的初始化方法，但并不启动WSGI服务；

        :param name: Pretty name for logging.
        :param app: The WSGI application to serve.
        :param host: IP address to serve the application.
        :param port: Port number to server the application.
        :param pool_size: Maximum number of eventlets to spawn concurrently.
        启动最大的线程的数目；
        :returns: None

        """
        self.name = name
        self.app = app
        self._host = host or "0.0.0.0"
        self._port = port or 0
        self._server = None
        self._socket = None
        self._protocol = protocol
        # GreenPool：绿色线程池；
        self._pool = eventlet.GreenPool(pool_size or self.default_pool_size)
        self._logger = logging.getLogger("eventlet.wsgi.server")
        self._wsgi_logger = logging.WritableLogger(self._logger)

可见在这里我们可以指定或者默认的建立具有若干绿色线程的绿色线程池。

我们再来看语句servers.append(service.Service.create(binary=binary))中的create方法，具体来看代码：

    def create(cls, host=None, binary=None, topic=None, manager=None,
               report_interval=None, periodic_interval=None,
               periodic_fuzzy_delay=None, service_name=None):
        """
        Instantiates class and passes back application object.

        :param host: defaults to CONF.host
        :param binary: defaults to basename of executable
        :param topic: defaults to bin_name - 'cinder-' part
        :param manager: defaults to CONF.<topic>_manager
        :param report_interval: defaults to CONF.report_interval
        :param periodic_interval: defaults to CONF.periodic_interval
        :param periodic_fuzzy_delay: defaults to CONF.periodic_fuzzy_delay

        """
        if not host:
            host = CONF.host
        if not binary:
            binary = os.path.basename(inspect.stack()[-1][1])
        if not topic:
            topic = binary
        if not manager:
            subtopic = topic.rpartition('cinder-')[2]
            manager = CONF.get('%s_manager' % subtopic, None)
        if report_interval is None:
            report_interval = CONF.report_interval
        if periodic_interval is None:
            periodic_interval = CONF.periodic_interval
        if periodic_fuzzy_delay is None:
            periodic_fuzzy_delay = CONF.periodic_fuzzy_delay
            
        service_obj = cls(host, binary, topic, manager,
                          report_interval=report_interval,
                          periodic_interval=periodic_interval,
                          periodic_fuzzy_delay=periodic_fuzzy_delay,
                          service_name=service_name)

可见方法create在进行了一系列变量的初始化操作后，应用这些变量作为参数进行类service.Service的初始化对象获取操作。

输出示例：

host = node01.shinian.com
binary = cinder-volume
topic = cinder-volume
manager = cinder.volume.manager.VolumeManager
report_interval = 10
periodic_interval = 60
periodic_fuzzy_delay = 60
cls = <class 'cinder.service.Service'>
service_obj = <cinder.service.Service object at 0x25ded90>

我们再回到脚本文件cinder-all中，来看语句service.serve(*servers)中的方法service，在这里把之前获取的三个服务类的初始化对象作为参数传入进来；

def serve(*servers):
    global _launcher
    if not _launcher:
        _launcher = Launcher()
    for server in servers:
        _launcher.launch_server(server)

来看语句_launcher.launch_server(server)中的方法launch_server：

    def launch_server(self, server):
        """
        Load and start the given server.
        加载并启动给定的服务；

        :param server: The server you would like to start.
        :returns: None
        """
        gt = eventlet.spawn(self.run_server, server)
        self._services.append(gt)

在这个方法中，有两个点需要关注，首先是方法spawn，以及方法self.run_server，首先来看方法spawn的实现：

spawn = greenthread.spawn

def spawn(func, *args, **kwargs):
    """
    Create a greenthread to run ``func(*args, **kwargs)``.  Returns a 
    :class:`GreenThread` object which you can use to get the results of the 
    call.
    
    Execution control returns immediately to the caller; the created greenthread
    is merely scheduled to be run at the next available opportunity.  
    Use :func:`spawn_after` to  arrange for greenthreads to be spawned 
    after a finite delay.
    """
    hub = hubs.get_hub()
    g = GreenThread(hub.greenlet)
    hub.schedule_call_global(0, g.switch, func, args, kwargs)
    return g

可见方法spawn实现的功能就是建立一个新的绿色线程用来运行变量func所指定的方法。

再来看方法self.run_server的实现：

def run_server(server):
        """
        Start and wait for a server to finish.
        启动并等待服务的结束；

        :param service: Server to run and wait for.
        :returns: None

        """
        server.start()
        server.wait()

在这里调用了server对应的start方法，来实现服务的启动，因为我们传进来的是分属两种不同类的实例化对象的三个服务，所以这里也对应的调用了不同类下的start方法；

首先来看cinder.service.WSGIService所对应的start方法的实现：

    def start(self):
        """
        Start serving this service using loaded configuration.
        应用加载的配置来启动指定的服务；

        Also, retrieve updated port number in case '0' was passed in, which
        indicates a random port should be used.

        :returns: None

        """
        if self.manager:
            self.manager.init_host()
        self.server.start()
        self.port = self.server.port

    def start(self, backlog=128):
        """
        Start serving a WSGI application.
        启动一个WSGI应用；

        :param backlog: Maximum number of queued connections.
        队列连接的最大数目；
        :returns: None
        :raises: cinder.exception.InvalidInput

        """
        if backlog < 1:
            raise exception.InvalidInput(
                reason='The backlog must be more than 1')

        # 获取连接的socket；
        self._socket = self._get_socket(self._host, self._port, backlog=backlog)
        self._server = eventlet.spawn(self._start)
        (self._host, self._port) = self._socket.getsockname()[0:2]
        LOG.info(_("Started %(name)s on %(host)s:%(port)s") %
                 {'name': self.name, 'host': self.host, 'port': self.port})

    def _start(self):
        """
        Run the blocking eventlet WSGI server.

        :returns: None

        """
        eventlet.wsgi.server(self._socket,
                             self.app,
                             protocol=self._protocol,
                             custom_pool=self._pool,
                             log=self._wsgi_logger)

我们可以看到，这里实现的就是启动了'osapi_volume'所指定的阻塞式eventlet WSGI服务。

我们再来看cinder.service.Service所对应的start方法的实现：

    def start(self):
        version_string = version.version_string()
        LOG.audit(_('Starting %(topic)s node (version %(version_string)s)'),
                  {'topic': self.topic, 'version_string': version_string})
        self.model_disconnected = False
        # get_admin_context：获取admin用户的上下文信息；
        ctxt = context.get_admin_context()
        try:
            # 通过主机节点名称和binary获取服务的状态信息；
            # 注：binary为指定的服务名称；
            service_ref = db.service_get_by_args(ctxt,
                                                 self.host,
                                                 self.binary)
            self.service_id = service_ref['id']
        except exception.NotFound:
            self._create_service_ref(ctxt)

        # create_connection：建立到用于rpc的信息总线的连接；
        # 建立一个新的连接，或者从连接池中获取一个连接；
        self.conn = rpc.create_connection(new=True)
        LOG.debug(_("Creating Consumer connection for Service %s") %
                  self.topic)

        # 为管理器获取RPC调度器；
        rpc_dispatcher = self.manager.create_rpc_dispatcher()

        # Share this same connection for these Consumers
        # 为消费者共享同样的连接；
        # 建立不同的消息消费者；
        # 创建以服务的topic为路由键的消费者；
        self.conn.create_consumer(self.topic, rpc_dispatcher, fanout=False)

        # 创建以服务的topic和本机名为路由键的消费者（基于topic&host，可用来接收定向消息）;
        node_topic = '%s:%s' % (self.topic, self.host)
        self.conn.create_consumer(node_topic, rpc_dispatcher, fanout=False)

        # fanout直接投递消息，不进行匹配，速度最快（fanout类型，可用于接收广播消息）；
        self.conn.create_consumer(self.topic, rpc_dispatcher, fanout=True)

        # Consume from all consumers in a thread
        # 启动消费者线程；
        # consume_in_thread用evelent.spawn创建一个协程一直运行；
        # 等待消息，在有消费到来时会创建新的协程运行远程调用的函数；
        self.conn.consume_in_thread()
        self.manager.init_host()

        if self.report_interval:
            pulse = utils.LoopingCall(self.report_state)
            pulse.start(interval=self.report_interval,
                        initial_delay=self.report_interval)
            self.timers.append(pulse)

        if self.periodic_interval:
            if self.periodic_fuzzy_delay:
                initial_delay = random.randint(0, self.periodic_fuzzy_delay)
            else:
                initial_delay = None

            periodic = utils.LoopingCall(self.periodic_tasks)
            periodic.start(interval=self.periodic_interval,
                           initial_delay=initial_delay)
            self.timers.append(periodic)

这个方法主要实现了获取所有服务、创建到RPC的连接，创建不同类型的消息消费者，启动消费者线程用来执行未来服务运行过程中所要获取到的消息。这个方法在我们之前的博客中已经进行过详细的解析，具体请看OpenStack建立实例完整过程源码详细分析（14）----依据AMQP通信架构实现消息接收机制解析之一 。

在这里我们就可以看到，因为我们这里启动了两个cinder.service.Service object类型的服务，即'cinder-volume'和'cinder-scheduler'，由上面的代码可以知道，每个服务都会启动和拥有属于自己的消息队列，用于处理和执行将要获取到的消息 。

最后我们再回到服务启动脚本文件cinder-all中，在程序的最后service.wait()语句中调用了方法wait，实现了等待所有服务运行结束或停止，然后关闭RPC协议框架，我们来看代码实现：

def wait():
    LOG.debug(_('Full set of CONF:'))
    for flag in CONF:
        flag_get = CONF.get(flag, None)
        # hide flag contents from log if contains a password
        # should use secret flag when switch over to openstack-common
        if ("_password" in flag or "_key" in flag or
                (flag == "sql_connection" and "mysql:" in flag_get)):
            LOG.debug(_('%s : FLAG SET ') % flag)
        else:
            LOG.debug('%(flag)s : %(flag_get)s' %
                      {'flag': flag, 'flag_get': flag_get})
    try:
        _launcher.wait()
    except KeyboardInterrupt:
        _launcher.stop()
    rpc.cleanup()

好啦，现在我们以cinder-all为例，简要的进行了cinder服务启动流程的解析，谢谢大家!!!!