openstack-nova-api源码剖析1

启动nova-api服务解析(rocky版本)

启动nova的api服务时，需要调用nova-api命令，nova-api命令最终是调用nova.cmd.api模块里的main方法。

nova-api命令的main方法解析:

def main():
#此方法会解析命令行传入进来的参数(sys.argv),配置文件的参数管理使用oslo.config模块进行管理。
    config.parse_args(sys.argv)
    #调用solo.log模块建立nova域的日志空间
    logging.setup(CONF, "nova")
    #注册所有需要的模块对象
    objects.register_all()
   #oslo_reports模块,用于生成错误报告,如内存泄漏等
    gmr_opts.set_defaults(CONF)
   
    if 'osapi_compute' in CONF.enabled_apis:
        # NOTE(mriedem): This is needed for caching the nova-compute service
        # version.
    #这里对于缓存nova-compute服务版本是必要的，当使用网络ID为'auto'或'none'，发出服务器创建请求时，将会查找该版本？（目前不理解）
        objects.Service.enable_min_version_cache()
    #构建获取logger
    log = logging.getLogger(__name__)
    #使用oslo.reports模块来管理收集错误报告
    gmr.TextGuruMeditation.setup_autorun(version, conf=CONF)
   #获取oslo.service模块的服务启动实例
    launcher = service.process_launcher()
    started = 0
   #定义了api-paste.ini两个默认的sections名,分别为osapi_compute,metadata
    for api in CONF.enabled_apis:
        should_use_ssl = api in CONF.enabled_ssl_apis
        try:
    #遍历启wsgi服务实例，会根据api-paste.ini配置文件加载app
            server = service.WSGIService(api, use_ssl=should_use_ssl)
            launcher.launch_service(server, workers=server.workers or 1)
            started += 1
        except exception.PasteAppNotFound as ex:
            log.warning("%s. ``enabled_apis`` includes bad values. "
                        "Fix to remove this warning.", ex)

    if started == 0:
        log.error('No APIs were started. '
                  'Check the enabled_apis config option.')
        sys.exit(1)
#服务阻塞
    launcher.wait()

api-paste.ini解析:

pasteDeploy通过文件配置的方式将各个功能以模块化的方式组装在一起。这样做的好处是把各个功能特性分块化后方便组装与拆卸，起到了松耦合的作用。

############
 # Metadata #
############
[composite:metadata]
use = egg:Paste#urlmap   #调用paste模块的urlmap来转发URL，配置文件中的enabled_apis的其中一个默认值就是metadata
/: meta    #将/的路径转发到[pipeline:meta]处理，:号也可以换成=号来映射

[pipeline:meta]
pipeline = cors metaapp

[app:metaapp]
paste.app_factory = nova.api.metadata.handler:MetadataRequestHandler.factory

#############
# OpenStack #
#############

[composite:osapi_compute] #这里的url路径映射是使用:号，也可以换成=号来代替
use = call:nova.api.openstack.urlmap:urlmap_factory #这里nova自定义了urlmap工厂来处理URL，配置文件中的enabled_apis的其中一个默认值就是osapi_compute
/: oscomputeversions         #将/的路径转发到[pipeline:oscomputeversions]处理
**v21是与v2完全匹配的功能，但它在wsgi表面上具有更严格的输入验证（防止API早期模糊化）。** 
**# 它还通过API微版本提供了新功能，这些微版本可供客户选择。不知情的客户端将接收到相同的冻结v2 API特性集，但有一些宽松的验证**

/v2: openstack_compute_api_v21_legacy_v2_compatible #将/v2开头的url转到[composite:openstack_compute_api_v21_legacy_v2_compatible]处理
/v2.1: openstack_compute_api_v21    #将/v2.1开头的url转到[composite:openstack_compute_api_v21]处理

[composite:openstack_compute_api_v21]
#调用自定义的composite工厂方法。方法里会根据配置文件的认证策略来决定加载下面哪一种应用,默认加载keystone,也可以将配置文件里的auth_strategy设置为noauth2。
use = call:nova.api.auth:pipeline_factory_v21
#在noauth2和keystone配置里不同的地方是noauth2和authtoken这两个认证filter,authtoken会先进行keystone登陆验证
noauth2 = cors http_proxy_to_wsgi compute_req_id faultwrap sizelimit osprofiler noauth2 osapi_compute_app_v21
keystone = cors http_proxy_to_wsgi compute_req_id faultwrap sizelimit osprofiler authtoken keystonecontext osapi_compute_app_v21

[composite:openstack_compute_api_v21_legacy_v2_compatible]
use = call:nova.api.auth:pipeline_factory_v21
noauth2 = cors http_proxy_to_wsgi compute_req_id faultwrap sizelimit osprofiler noauth2 legacy_v2_compatible osapi_compute_app_v21
keystone = cors http_proxy_to_wsgi compute_req_id faultwrap sizelimit osprofiler authtoken keystonecontext legacy_v2_compatible osapi_compute_app_v21

[filter:request_id]
# 它确保为每个API请求分配请求ID并将其设置到request environment。请求ID也被添加到API响应中。 
paste.filter_factory = oslo_middleware:RequestId.factory # 确保请求ID的中间件

[filter:compute_req_id]
# 它确保为每个请求到compute服务的api分配请求ID并将其设置到request environ。请求ID也被添加到API响应中。
paste.filter_factory = nova.api.compute_req_id:ComputeReqIdMiddleware.factory

[filter:faultwrap]
paste.filter_factory = nova.api.openstack:FaultWrapper.factory

[filter:noauth2]
paste.filter_factory = nova.api.openstack.auth:NoAuthMiddleware.factory

[filter:osprofiler]
paste.filter_factory = nova.profiler:WsgiMiddleware.factory

[filter:sizelimit]
paste.filter_factory = oslo_middleware:RequestBodySizeLimiter.factory #限制传入请求的大小的中间件。 

[filter:http_proxy_to_wsgi]
# 此中间件使用远程HTTP反向代理提供的环境变量重载WSGI环境变量。 
paste.filter_factory = oslo_middleware.http_proxy_to_wsgi:HTTPProxyToWSGI.factory # 到WSGI终止中间件的HTTP代理。 

[filter:legacy_v2_compatible]
paste.filter_factory = nova.api.openstack:LegacyV2CompatibleWrapper.factory

[app:osapi_compute_app_v21]
paste.app_factory = nova.api.openstack.compute:APIRouterV21.factory

[pipeline:oscomputeversions]
pipeline = cors faultwrap http_proxy_to_wsgi oscomputeversionapp

[app:oscomputeversionapp]
paste.app_factory = nova.api.openstack.compute.versions:Versions.factory

##########
# Shared #
##########

[filter:cors]
# 这个中间件允许WSGI应用程序为多个配置的域提供CORS报头.
paste.filter_factory = oslo_middleware.cors:filter_factory # CORS中间件 
oslo_config_project = nova

[filter:keystonecontext]
paste.filter_factory = nova.api.auth:NovaKeystoneContext.factory

[filter:authtoken]
paste.filter_factory = keystonemiddleware.auth_token:filter_factory

方法调用关系分析：
缓存版本实现方法：作用目前不明
classmethod 修饰符对应的函数不需要实例化，不需要 self 参数，但第一个参数需要是表示自身类的 cls 参数，可以来调用类的属性，类的方法，实例化对象等。

模块服务进程启动：
Openstack中有一个叫Launcher的概念，即专门用来启动服务的，这个类被放在了oslo_service这个包里面。Launcher分为两种，一种是ServiceLauncher，另一种为ProcessLauncher。ServiceLauncher用来启动单进程的服务，而ProcessLauncher用来启动有多个worker子进程的服务。
ProcessLauncher直接继承于Object，所以其对launch_service方法进行了实现。后面会详细介绍。

核心代码：
server = service.WSGIService(api, use_ssl=should_use_ssl)
launcher.launch_service(server, workers=server.workers or 1)

WSGIService的定义在nova/service.py文件中，代码如下:
根据传入的参数name创建一个wsgi server，首先通过
wsgi.Loader.load_app(name)载入app。
通过Python库中的deploy.loadapp加载在文件/etc/nova/api-paste.ini中定义的osapi_compute，如下所以，v1版本的api请求由oscomputeversions处理，v2版本的请求由openstack_compute_api_v21_legacy_v2_compatible处理，v2.1版本的api请求由openstack_compute_api_v21处理。

#############
# OpenStack #
#############

[composite:osapi_compute]
use = call:nova.api.openstack.urlmap:urlmap_factory
/: oscomputeversions
# v21 is an exactly feature match for v2, except it has more stringent
# input validation on the wsgi surface (prevents fuzzing early on the
# API). It also provides new features via API microversions which are
# opt into for clients. Unaware clients will receive the same frozen
# v2 API feature set, but with some relaxed validation
/v2: openstack_compute_api_v21_legacy_v2_compatible
/v2.1: openstack_compute_api_v21

[composite:openstack_compute_api_v21]
use = call:nova.api.auth:pipeline_factory_v21
noauth2 = cors http_proxy_to_wsgi compute_req_id faultwrap request_log sizelimit osprofiler noauth2 osapi_compute_app_v21
keystone = cors http_proxy_to_wsgi compute_req_id faultwrap request_log sizelimit osprofiler authtoken keystonecontext osapi_compute_app_v21

[composite:openstack_compute_api_v21_legacy_v2_compatible]
use = call:nova.api.auth:pipeline_factory_v21
noauth2 = cors http_proxy_to_wsgi compute_req_id faultwrap request_log sizelimit osprofiler noauth2 legacy_v2_compatible osapi_compute_app_v21
keystone = cors http_proxy_to_wsgi compute_req_id faultwrap request_log sizelimit osprofiler authtoken keystonecontext legacy_v2_compatible osapi_compute_app_v21

回到nova/service.py文件中，载入app以后，根据配置文件设置的IP和端口，设置app的监听端口

然后创建一个wsgi类的Server，代码如下：

class Server(service.ServiceBase):
    """Server class to manage a WSGI server, serving a WSGI application."""

    default_pool_size = CONF.wsgi.default_pool_size

    def __init__(self, name, app, host='0.0.0.0', port=0, pool_size=None,
                       protocol=eventlet.wsgi.HttpProtocol, backlog=128,
                       use_ssl=False, max_url_len=None):
        """Initialize, but do not start, a WSGI server.
        :param name: Pretty name for logging.
        :param app: The WSGI application to serve.
        :param host: IP address to serve the application.
        :param port: Port number to server the application.
        :param pool_size: Maximum number of eventlets to spawn concurrently.
        :param backlog: Maximum number of queued connections.
        :param max_url_len: Maximum length of permitted URLs.
        :returns: None
        :raises: nova.exception.InvalidInput
        """
        # Allow operators to customize http requests max header line size.
        eventlet.wsgi.MAX_HEADER_LINE = CONF.wsgi.max_header_line
        self.name = name
        self.app = app
        self._server = None
        self._protocol = protocol
        self.pool_size = pool_size or self.default_pool_size
        self._pool = eventlet.GreenPool(self.pool_size)
        self._logger = logging.getLogger("nova.%s.wsgi.server" % self.name)
        self._use_ssl = use_ssl
        self._max_url_len = max_url_len
        self.client_socket_timeout = CONF.wsgi.client_socket_timeout or None

        if backlog < 1:
            raise exception.InvalidInput(
                    reason=_('The backlog must be more than 0'))

        bind_addr = (host, port)
        # TODO(dims): eventlet's green dns/socket module does not actually
        # support IPv6 in getaddrinfo(). We need to get around this in the
        # future or monitor upstream for a fix
        try:
            info = socket.getaddrinfo(bind_addr[0],
                                      bind_addr[1],
                                      socket.AF_UNSPEC,
                                      socket.SOCK_STREAM)[0]
            family = info[0]
            bind_addr = info[-1]
        except Exception:
            family = socket.AF_INET

        try:
            self._socket = eventlet.listen(bind_addr, family, backlog=backlog)
        except EnvironmentError:
            LOG.error("Could not bind to %(host)s:%(port)s",
                      {'host': host, 'port': port})
            raise

        (self.host, self.port) = self._socket.getsockname()[0:2]
        LOG.info("%(name)s listening on %(host)s:%(port)s",
                 {'name': self.name, 'host': self.host, 'port': self.port})

主要的代码为self._socket = eventlet.listen(bind_addr, family, backlog=backlog)，创建一个socket监听，至此nova/cmd/api.py中的核心代码server = service.WSGIService(api, use_ssl=should_use_ssl)执行完毕了，接下来执行

launcher.launch_service(server, workers=server.workers or 1)

launcher = service.process_launcher() 代码定义在nova/service.py中

def process_launcher():
return service.ProcessLauncher(CONF, restart_method=‘mutate’)

类ProcessLauncher在文件oslo_service/service.py中定义，launch_service的代码如下：

class ProcessLauncher(object):
    ...
    def launch_service(self, service, workers=1):
        """Launch a service with a given number of workers.

       :param service: a service to launch, must be an instance of
              :class:`oslo_service.service.ServiceBase`
       :param workers: a number of processes in which a service
              will be running
        """
        _check_service_base(service)
        wrap = ServiceWrapper(service, workers)

        LOG.info('Starting %d workers', wrap.workers)
        while self.running and len(wrap.children) < wrap.workers:
            self._start_child(wrap)

lauch_service除了接受service参数以外，还需要接受一个workers参数，即子进程的个数，然后调用_start_child启动多个子进程。

553     def _start_child(self, wrap):
554         if len(wrap.forktimes) > wrap.workers:
555             # Limit ourselves to one process a second (over the period of
556             # number of workers * 1 second). This will allow workers to
557             # start up quickly but ensure we don't fork off children that
558             # die instantly too quickly.
559             if time.time() - wrap.forktimes[0] < wrap.workers:
560                 LOG.info('Forking too fast, sleeping')
561                 time.sleep(1)
562 
563             wrap.forktimes.pop(0)
564 
565         wrap.forktimes.append(time.time())
566 
567         pid = os.fork()
568         if pid == 0:
569             self.launcher = self._child_process(wrap.service)
570             while True:
571                 self._child_process_handle_signal()
572                 status, signo = self._child_wait_for_exit_or_signal(
573                     self.launcher)
574                 if not _is_sighup_and_daemon(signo):
575                     self.launcher.wait()
576                     break
577                 self.launcher.restart()
578 
579             os._exit(status)
580 
581         LOG.debug('Started child %d', pid)
582 
583         wrap.children.add(pid)
584         self.children[pid] = wrap
585 
586         return pid

_start_child只是简单的调用了一个os.fork()，然后子进程开始运行，子进程调用_child_process。

534     def _child_process(self, service):
535         self._child_process_handle_signal()
536 
537         # Reopen the eventlet hub to make sure we don't share an epoll
538         # fd with parent and/or siblings, which would be bad
539         eventlet.hubs.use_hub()
540 
541         # Close write to ensure only parent has it open
542         os.close(self.writepipe)
543         # Create greenthread to watch for parent to close pipe
544         eventlet.spawn_n(self._pipe_watcher)
545 
546         # Reseed random number generator
547         random.seed()
548        #类Launcher下的launch_service方法
549         launcher = Launcher(self.conf, restart_method=self.restart_method)
550         launcher.launch_service(service)
551         return launcher

_child_process其实很简单，创建一个Launcher，调用Laucher.launch_service方法，调用service.start方法启动服务。

271     def launch_service(self, service, workers=1):
272         """Load and start the given service.
273 
274         :param service: The service you would like to start, must be an
275                         instance of :class:`oslo_service.service.ServiceBase`
276         :param workers: This param makes this method compatible with
277                         ProcessLauncher.launch_service. It must be None, 1 or
278                         omitted.
279         :returns: None
280 
281         """
282         if workers is not None and workers != 1:
283             raise ValueError(_("Launcher asked to start multiple workers"))
284         _check_service_base(service)
285         service.backdoor_port = self.backdoor_port
286         self.services.add(service)

laucher_service就是将服务添加到self.services成员里面，services成员的类型是class Services，看看它的add方法。

744 class Services(object):
745 
746     def __init__(self):
747         self.services = []
748         self.tg = threadgroup.ThreadGroup()
749         self.done = event.Event()
750 
751     def add(self, service):
752         """Add a service to a list and create a thread to run it.
753 
754         :param service: service to run
755         """
756         self.services.append(service)
757         self.tg.add_thread(self.run_service, service, self.done)
758         ...
786     @staticmethod
787     def run_service(service, done):
788         """Service start wrapper.
789 
790         :param service: service to run
791         :param done: event to wait on until a shutdown is triggered
792         :returns: None
793 
794         """
795         try:
796             service.start()
797         except Exception:
798             LOG.exception('Error starting thread.')
799             raise SystemExit(1)
800         else:
801             done.wait()

oslo_service/threadgroup.py

    def add_thread(self, callback, *args, **kwargs):
        """Spawn a new thread.
        This call will block until capacity is available in the thread pool.
        After that, it returns immediately (i.e. *before* the new thread is
        scheduled).
        :param callback: the function to run in the new thread.
        :param args: positional arguments to the callback function.
        :param kwargs: keyword arguments to the callback function.
        :returns: a :class:`Thread` object
        """
        gt = self.pool.spawn(callback, *args, **kwargs)
        th = Thread(gt, self, link=False)
        self.threads.append(th)
        gt.link(_on_thread_done, self, th)
        return th

Services这个类的初始化很简单，即创建一个ThreadGroup，ThreadGroup其实是eventlet的GreenPool，Openstack利用eventlet实现并发。add方法，将self.run_service这个方法放入pool中，而service就是它的参数。run_service方法很简单，就是调用service的start方法，这样就完成了服务的启动。
至此，api.py启动完毕。

方法调用关系：

nova/cmd/api.py  nova/objects/service.py    nova/objects/service.py   nova/objects/service.py
main()->enable_min_version_cache()->clear_min_version_cache()->_MIN_VERSION_CACHE = {}
                                  ->_SERVICE_VERSION_CACHING = True(初始为False)
        nova/service.py   site-packages/oslo_service/service.py
      ->process_launcher()->ProcessLauncher()
        nova/service.py
      ->WSGIService()
            nova/api/wsgi.py
         ->loader.load_app(name) 
            ->deploy.loadapp() 
             #通过Python库中的deploy.loadapp加载在文件/etc/nova/api-paste.ini中定义的osapi_compute，如下所以，v1版本的api请求由#oscomputeversions处理，v2版本的请求由openstack_compute_api_v21_legacy_v2_compatible处理，v2.1版本的api请求由#openstack_compute_api_v21处理。
              回到nova/service.py文件中，载入app以后，根据配置文件设置的IP和端口，设置app的监听端口，然后创建一个wsgi类的Server:
            nova/wsgi.py
          ->wsgi.Server(name,self.app,host=self.host,port=self.port,use_ssl=self.use_ssl,max_url_len=max_url_len)
            ->._socket = eventlet.listen()#创建一个socket监听
        #launcher = service.process_launcher() 代码定义在nova/service.py中,
        #def process_launcher():
           #return service.ProcessLauncher(CONF, restart_method='mutate')类ProcessLauncher在文件oslo_service/service.py中定义
      ->launcher.launch_service(server, workers=server.workers or 1) 
           #lauch_service除了接受service参数以外，还需要接受一个workers参数，即子进程的个数，然后调用_start_child启动多个子进程。
        ->_start_child(wrap)
           #_start_child只是简单的调用了一个os.fork()，然后子进程开始运行，子进程调用_child_process。
          ->_child_process()
            ->launcher = Launcher()
            ->launcher.launch_service(service)
              ->self.services.add(service)
                  oslo_service/threadgroup.py
                ->self.tg.add_thread(self.run_service, service, self.done)
                ->self.run_service
                  ->service.start()

 
 
 
 
 

参考：https://blog.csdn.net/weixin_33813128/article/details/91949476
参考：https://blog.csdn.net/lemonwsx/article/details/84028373
参考：https://www.cnblogs.com/Luka-Modric/p/8081648.html