关于opentack的(3)--nova api简析

wsgi参考:
paste
http://www.cnblogs.com/persevere/p/3611958.html
http://www.cnblogs.com/Security-Darren/p/4087587.html

Webob
http://www.cnblogs.com/lxguidu/archive/2013/04/25/3042079.html
config参考:
http://blog.csdn.net/alvine008/article/details/24364243

我们以上篇nova list命令执行过程的api调用为线索,探索openstack nova模块的api初始化和调用请求响应.
nova list命令涉及两次api调用,一次是是keystone的,一次是nova的,keystone的我们先不管,我们以nova api的调用为线索.
访问的nova api为(我的计算节点ip为:192.168.16.190)
http://192.168.16.190:8774/v2/6ff51bde9ba940e8b90892988a0c3a82/servers/detail
下面我们去探索,这个api是怎么建立起来的.
执行如下命令,得知监听8774端口的进程id为17630:

root@os:~/openshit# netstat -apn | grep 8774
tcp        0      0 0.0.0.0:8774            0.0.0.0:*               LISTEN      17630/python

接着找出进程id为17630的进程:

root@os:~/openshit# ps ax | grep 17630
  440 pts/1    S+     0:00 grep --color=auto 17630
17630 ?        Ss     6:53 /usr/bin/python /usr/bin/nova-api --config-file=/etc/nova/nova.conf

可知nova api的入口函数就在/usr/bin/nova-api(而且使用配置文件在/etc/nova/nova.conf):

#! /usr/bin/python
# PBR Generated from u'console_scripts'

import sys

from nova.cmd.api import main


if __name__ == "__main__":
    sys.exit(main())

原来是个马甲,我们去\usr\lib\python2.7\dist-packages\nova\cmd\api.py:

import sys

from oslo.config import cfg

from nova import config
from nova import objects
from nova.openstack.common import log as logging
from nova.openstack.common.report import guru_meditation_report as gmr
from nova import service
from nova import utils
from nova import version

CONF = cfg.CONF
CONF.import_opt('enabled_apis', 'nova.service')
CONF.import_opt('enabled_ssl_apis', 'nova.service')


def main():
    #解析输入参数，填充到CONF中
    config.parse_args(sys.argv)
    logging.setup("nova")
    utils.monkey_patch()
    objects.register_all()

    gmr.TextGuruMeditation.setup_autorun(version)

    launcher = service.process_launcher()
    #一般/etv/nova/nova.conf有enabled_apis=ec2,osapi_compute,metadata三种api,然后去/etc/nova/api-paste.ini获取
    for api in CONF.enabled_apis:
        should_use_ssl = api in CONF.enabled_ssl_apis
        if api == 'ec2':
            server = service.WSGIService(api, use_ssl=should_use_ssl,
                                         max_url_len=16384)
        else:
            server = service.WSGIService(api, use_ssl=should_use_ssl)
        launcher.launch_service(server, workers=server.workers or 1)
    launcher.wait()

看到将解析/etc/nova/nova.conf,填充到CONF中. CONF.enabled_apis这个是使能的API类型,openstack支持三种API,ec2,osapi和metadata.去看配置文件/etc/nova/nova.conf可以看到这么一段:

enabled_apis=ec2,osapi_compute,metadata

默认是三种api都启用的,可以自行配置启用哪些api.这里我们研究osapi(也就是 openstack api).api=’osapi_compute’作为参数调用WSGIService就是初始化osapi. WSGIService定义在nova\service.py:

class WSGIService(object):
    """Provides ability to launch API from a 'paste' configuration."""

    def __init__(self, name, loader=None, use_ssl=False, max_url_len=None):
        """Initialize, but do not start the WSGI server.

        :param name: The name of the WSGI server given to the loader.
        :param loader: Loads the WSGI application using the given name.
        :returns: None

        """
        self.name = name
        self.manager = self._get_manager()
        #并没有传入loader
        self.loader = loader or wsgi.Loader()
        #根据名字加载/etc/nova/api-paste.ini中定义的对应app
        self.app = self.loader.load_app(name)
        #监听ip
        self.host = getattr(CONF, '%s_listen' % name, "0.0.0.0")
        #监听端口,openstack的api默认8774,就在本文件的上面定义
        self.port = getattr(CONF, '%s_listen_port' % name, 0)
        self.workers = (getattr(CONF, '%s_workers' % name, None) or
                        processutils.get_worker_count())
        if self.workers and self.workers < 1:
            worker_name = '%s_workers' % name
            msg = (_("%(worker_name)s value of %(workers)s is invalid, "
                     "must be greater than 0") %
                   {'worker_name': worker_name,
                    'workers': str(self.workers)})
            raise exception.InvalidInput(msg)
        self.use_ssl = use_ssl
        self.server = wsgi.Server(name,
                                  self.app,
                                  host=self.host,
                                  port=self.port,
                                  use_ssl=self.use_ssl,
                                  max_url_len=max_url_len)
        # Pull back actual port used
        self.port = self.server.port
        self.backdoor_port = None

WSGIService是个类,server = service.WSGIService(api, use_ssl=should_use_ssl)生成一个WSGIService对象server,将调用上面的初始化函数.看到:

        self.manager = self._get_manager()
        #并没有传入loader
        self.loader = loader or wsgi.Loader()

没有传入loader,所以将会使用wsgi.Loader()返回的loader:

class Loader(object):
    """Used to load WSGI applications from paste configurations."""

    def __init__(self, config_path=None):
        """Initialize the loader, and attempt to find the config.

        :param config_path: Full or relative path to the paste config.
        :returns: None

        """
        self.config_path = None

        config_path = config_path or CONF.api_paste_config
        if not os.path.isabs(config_path):
            self.config_path = CONF.find_file(config_path)
        elif os.path.exists(config_path):
            self.config_path = config_path

        if not self.config_path:
            raise exception.ConfigNotFound(path=config_path)

    def load_app(self, name):
        """Return the paste URLMap wrapped WSGI application.

        :param name: Name of the application to load.
        :returns: Paste URLMap object wrapping the requested application.
        :raises: `nova.exception.PasteAppNotFound`

        """
        try:
            LOG.debug("Loading app %(name)s from %(path)s",
                      {'name': name, 'path': self.config_path})
            return deploy.loadapp("config:%s" % self.config_path, name=name)
        except LookupError as err:
            LOG.error(err)
            raise exception.PasteAppNotFound(name=name, path=self.config_path)

看到如果用load_app载入wsgi应用将使用self.config_path的配置文件.那self.config_path是什么呢?被赋值为CONF.api_paste_config.我们去看下nov的文件文件/etc/nova/nova.conf有:

api_paste_config=/etc/nova/api-paste.ini

好了这下明了了,回到WSGIService的初始化函数,WSGIService构造一个Loader对象,里Loader对象中的loader从paste配置文件/etc/nova/api-paste.ini载入使能的API(ec2,osapi_compute,metadata,这里我们以osapi_compute为重点),看下api-paste.ini:

############
# Metadata #
############
[composite:metadata]
use = egg:Paste#urlmap
/: meta

[pipeline:meta]
pipeline = ec2faultwrap logrequest metaapp

[app:metaapp]
paste.app_factory = nova.api.metadata.handler:MetadataRequestHandler.factory

#######
# EC2 #
#######

[composite:ec2]
use = egg:Paste#urlmap
/services/Cloud: ec2cloud

[composite:ec2cloud]
use = call:nova.api.auth:pipeline_factory
noauth = ec2faultwrap logrequest ec2noauth cloudrequest validator ec2executor
keystone = ec2faultwrap logrequest ec2keystoneauth cloudrequest validator ec2executor

[filter:ec2faultwrap]
paste.filter_factory = nova.api.ec2:FaultWrapper.factory

[filter:logrequest]
paste.filter_factory = nova.api.ec2:RequestLogging.factory

[filter:ec2lockout]
paste.filter_factory = nova.api.ec2:Lockout.factory

[filter:ec2keystoneauth]
paste.filter_factory = nova.api.ec2:EC2KeystoneAuth.factory

[filter:ec2noauth]
paste.filter_factory = nova.api.ec2:NoAuth.factory

[filter:cloudrequest]
controller = nova.api.ec2.cloud.CloudController
paste.filter_factory = nova.api.ec2:Requestify.factory

[filter:authorizer]
paste.filter_factory = nova.api.ec2:Authorizer.factory

[filter:validator]
paste.filter_factory = nova.api.ec2:Validator.factory

[app:ec2executor]
paste.app_factory = nova.api.ec2:Executor.factory

#############
# OpenStack #
#############
#load osapi_compute将找到这里
#有5个分支,到时根据访问的url将交付给不同的wsgi app处理
#nova list命令涉及的api是以/v2开头的,所以请求会被openstack_compute_api_v2处理
#为何会这样?这个是use = call:nova.api.openstack.urlmap:urlmap_factory这句决定的
#这里我们不深究
[composite:osapi_compute]
use = call:nova.api.openstack.urlmap:urlmap_factory
/: oscomputeversions
/v1.1: openstack_compute_api_v2
/v2: openstack_compute_api_v2
/v2.1: openstack_compute_api_v21
/v3: openstack_compute_api_v3

#以/v2开头的请求将会在这里处理
#这里有三个分支:noauth ,keystone ,keystone_nolimit ,到底走哪个分支,这个是由
#use = call:nova.api.auth:pipeline_factory决定的
#对于nova list命令将走keystone_nolimit 流程

[composite:openstack_compute_api_v2]
use = call:nova.api.auth:pipeline_factory
noauth = compute_req_id faultwrap sizelimit noauth ratelimit osapi_compute_app_v2
keystone = compute_req_id faultwrap sizelimit authtoken keystonecontext ratelimit osapi_compute_app_v2
keystone_nolimit = compute_req_id faultwrap sizelimit authtoken keystonecontext osapi_compute_app_v2

[composite:openstack_compute_api_v21]
use = call:nova.api.auth:pipeline_factory_v21
noauth = request_id faultwrap sizelimit noauth osapi_compute_app_v21
keystone = request_id faultwrap sizelimit authtoken keystonecontext osapi_compute_app_v21

[composite:openstack_compute_api_v3]
use = call:nova.api.auth:pipeline_factory_v21
noauth = request_id faultwrap sizelimit noauth_v3 osapi_compute_app_v3
keystone = request_id faultwrap sizelimit authtoken keystonecontext osapi_compute_app_v3

[filter:request_id]
paste.filter_factory = nova.openstack.common.middleware.request_id:RequestIdMiddleware.factory

[filter:compute_req_id]
paste.filter_factory = nova.api.compute_req_id:ComputeReqIdMiddleware.factory

[filter:faultwrap]
paste.filter_factory = nova.api.openstack:FaultWrapper.factory

[filter:noauth]
paste.filter_factory = nova.api.openstack.auth:NoAuthMiddleware.factory

[filter:noauth_v3]
paste.filter_factory = nova.api.openstack.auth:NoAuthMiddlewareV3.factory

[filter:ratelimit]
paste.filter_factory = nova.api.openstack.compute.limits:RateLimitingMiddleware.factory

[filter:sizelimit]
paste.filter_factory = nova.api.sizelimit:RequestBodySizeLimiter.factory
#流程走到这里wsgi的任务也就完成了
#接着进入nova.api.openstack.compute:APIRouter.factory
[app:osapi_compute_app_v2]
paste.app_factory = nova.api.openstack.compute:APIRouter.factory

[app:osapi_compute_app_v21]
paste.app_factory = nova.api.openstack.compute:APIRouterV21.factory

[app:osapi_compute_app_v3]
paste.app_factory = nova.api.openstack.compute:APIRouterV3.factory

[pipeline:oscomputeversions]
pipeline = faultwrap oscomputeversionapp

[app:oscomputeversionapp]
paste.app_factory = nova.api.openstack.compute.versions:Versions.factory

##########
# Shared #
##########

[filter:keystonecontext]
paste.filter_factory = nova.api.auth:NovaKeystoneContext.factory

[filter:authtoken]
paste.filter_factory = keystonemiddleware.auth_token:filter_factory

看上面的配置和注释,回忆nova list调用的api:
http://192.168.16.190:8774/v2/6ff51bde9ba940e8b90892988a0c3a82/servers/detail
看下请求的处理流程:
初始化时加载名为osapi_compute
/v2开头,将交给openstack_compute_api_v2, openstack_compute_api_v2有三个分支(noauth ,keystone ,keystone_nolimit), nova.api.auth:pipeline_factory将根据请求的参数决定走什么流程,pipeline_factory函数实现在:nova\api\auth.py:

auth_opts = [
    cfg.BoolOpt('api_rate_limit',
                default=False,
                help=('Whether to use per-user rate limiting for the api. '
                      'This option is only used by v2 api. Rate limiting '
                      'is removed from v3 api.')),
    cfg.StrOpt('auth_strategy',
               default='keystone',
               help='The strategy to use for auth: noauth or keystone.'),
    cfg.BoolOpt('use_forwarded_for',
                default=False,
                help='Treat X-Forwarded-For as the canonical remote address. '
                     'Only enable this if you have a sanitizing proxy.'),
]

CONF = cfg.CONF
CONF.register_opts(auth_opts)

......


def pipeline_factory(loader, global_conf, **local_conf):
    """A paste pipeline replica that keys off of auth_strategy."""
    #默认是keystone,就在上面定义
    pipeline = local_conf[CONF.auth_strategy]
    LOG.info('auth_strategy:%s'%CONF.auth_strategy )
    #默认是false
    if not CONF.api_rate_limit:
        limit_name = CONF.auth_strategy + '_nolimit'
        pipeline = local_conf.get(limit_name, pipeline)
    pipeline = pipeline.split()
    # NOTE (Alex Xu): This is just for configuration file compatibility.
    # If the configuration file still contains 'ratelimit_v3', just ignore it.
    # We will remove this code at next release (J)
    if 'ratelimit_v3' in pipeline:
        LOG.warn(_LW('ratelimit_v3 is removed from v3 api.'))
        pipeline.remove('ratelimit_v3')
    return _load_pipeline(loader, pipeline)

上面的代码和注释不难看出将走分支keystone_nolimit,也就是:

keystone_nolimit = compute_req_id faultwrap sizelimit authtoken keystonecontext osapi_compute_app_v2

经过一层层处理(过滤),符合条件的将交给osapi_compute_app_v2处理,从配置文件api-paste.ini从看出,也就是交到nova.api.openstack.compute:APIRouter.factory,定义在文件nova\api\openstack\compute__init__.py:

class APIRouter(nova.api.openstack.APIRouter):
    """Routes requests on the OpenStack API to the appropriate controller
    and method.
    """
    ExtensionManager = extensions.ExtensionManager

    def _setup_routes(self, mapper, ext_mgr, init_only):
        if init_only is None or 'versions' in init_only:
            self.resources['versions'] = versions.create_resource()
            mapper.connect("versions", "/",
                        controller=self.resources['versions'],
                        action='show',
                        conditions={"method": ['GET']})

        mapper.redirect("", "/")
        """
       member: 是对单个实体进行操作,创建路由格式是:/controller/id/your_method
       collection:是对实体集合进行操作,创建路由格式是:/controller/your_method
       new:是创建一个实体,创建路由格式是:/controller/your_method/new

       """
        if init_only is None or 'consoles' in init_only:
            self.resources['consoles'] = consoles.create_resource()
            mapper.resource("console", "consoles",
                        controller=self.resources['consoles'],
                        parent_resource=dict(member_name='server',
                        collection_name='servers'))

        if init_only is None or 'consoles' in init_only or \
                'servers' in init_only or 'ips' in init_only:
            self.resources['servers'] = servers.create_resource(ext_mgr)
            #/servers/detail
            mapper.resource("server", "servers",
                            controller=self.resources['servers'],
                            collection={'detail': 'GET'},
                            member={'action': 'POST'})

        if init_only is None or 'ips' in init_only:
            self.resources['ips'] = ips.create_resource()
            mapper.resource("ip", "ips", controller=self.resources['ips'],
                            parent_resource=dict(member_name='server',
                                                 collection_name='servers'))

        if init_only is None or 'images' in init_only:
            self.resources['images'] = images.create_resource()
            mapper.resource("image", "images",
                            controller=self.resources['images'],
                            collection={'detail': 'GET'})

        if init_only is None or 'limits' in init_only:
            self.resources['limits'] = limits.create_resource()
            mapper.resource("limit", "limits",
                            controller=self.resources['limits'])

        if init_only is None or 'flavors' in init_only:
            self.resources['flavors'] = flavors.create_resource()
            mapper.resource("flavor", "flavors",
                            controller=self.resources['flavors'],
                            collection={'detail': 'GET'},
                            member={'action': 'POST'})

        if init_only is None or 'image_metadata' in init_only:
            self.resources['image_metadata'] = image_metadata.create_resource()
            image_metadata_controller = self.resources['image_metadata']

            mapper.resource("image_meta", "metadata",
                            controller=image_metadata_controller,
                            parent_resource=dict(member_name='image',
                            collection_name='images'))

            mapper.connect("metadata",
                           "/{project_id}/images/{image_id}/metadata",
                           controller=image_metadata_controller,
                           action='update_all',
                           conditions={"method": ['PUT']})

        if init_only is None or 'server_metadata' in init_only:
            self.resources['server_metadata'] = \
                server_metadata.create_resource()
            server_metadata_controller = self.resources['server_metadata']

            mapper.resource("server_meta", "metadata",
                            controller=server_metadata_controller,
                            parent_resource=dict(member_name='server',
                            collection_name='servers'))

            mapper.connect("metadata",
                           "/{project_id}/servers/{server_id}/metadata",
                           controller=server_metadata_controller,
                           action='update_all',
                           conditions={"method": ['PUT']})

聪明的你应该发现

            mapper.resource("server", "servers",
                            controller=self.resources['servers'],
                            collection={'detail': 'GET'},
                            member={'action': 'POST'})

这个与nova list请求的api是有关系的.