感谢朋友支持本博客,欢迎共同探讨交流,由于能力和时间有限,错误之处在所难免,欢迎指正!
如果转载,请保留作者信息。
博客地址:http://blog.csdn.net/gaoxingnengjisuan
邮箱地址:dong.liu@siat.ac.cn
PS:最近没有登录博客,很多朋友的留言没有看见,这里道歉!还有就是本人较少上QQ,可以邮件交流。
概述:
这篇博客关注的问题就是:
请求信息如何通过swift-proxy服务实现转发;
这里应用网上找到的一幅图片来看看swift-proxy服务在swift系统中的作用(多谢作者);
简单说,请求过来之后,通过swift-proxy服务的服务入口点,调用swift-proxy服务上的相关方法,实现通过给定的请求来确定具体要获取的控制器(ObjectController、ContainerController、AccountController),并进一步调用控制器下的具体方法,实现到相应存储服务(Object-server、Container-server、Account-server)的连接,并具体调用相应的方法来实现请求;
下面来看看代码的实现:
- def __call__(self, env, start_response):
- """
- 调用WSGI服务的入口点;
- """
- try:
- # 获取缓存信息;
- # 在env中获取'swift.cache'的值,如果为none,则报错误提示;
- # 注:env:表示wsgi环境的字典;
- if self.memcache is None:
- self.memcache = cache_from_env(env)
- req = self.update_request(Request(env))
- return self.handle_request(req)(env, start_response)
- except UnicodeError:
- err = HTTPPreconditionFailed(request=req, body='Invalid UTF8 or contains NULL')
- return err(env, start_response)
- except (Exception, Timeout):
- start_response('500 Server Error',[('Content-Type', 'text/plain')])
- return ['Internal server error.\n']
- def handle_request(self, req):
- """
- 代理服务的入口点;
- 对HTTP请求进行具体的处理和执行等操作;
- """
- try:
- self.logger.set_statsd_prefix('proxy-server')
- if req.content_length and req.content_length < 0:
- self.logger.increment('errors')
- return HTTPBadRequest(request=req, body='Invalid Content-Length')
- try:
- if not check_utf8(req.path_info):
- self.logger.increment('errors')
- return HTTPPreconditionFailed(request=req, body='Invalid UTF8 or contains NULL')
- except UnicodeError:
- self.logger.increment('errors')
- return HTTPPreconditionFailed(request=req, body='Invalid UTF8 or contains NULL')
- try:
- # 根据给定的HTTP的请求路径path获取控制器来处理请求;
- # 如果account, container, object都存在,则获取ObjectController控制器;
- # 如果account, container存在,object不存在,则获取ContainerController控制器;
- # 如果account存在,container, object不存在,则获取AccountController控制器;
- # 返回值:
- # path_parts:version, account, container, object等值的字典;
- # controller:具体的控制器类实例化对象;
- controller, path_parts = self.get_controller(req.path)
- # req.path = /v1/AUTH_2a8cbfbb8ad7411c8465f57311527937/testcontainer2/ceph9
- # controller = <class 'swift.proxy.controllers.obj.ObjectController'>
- # path_parts = {'object_name': 'ceph9', 'version': 'v1',
- # 'account_name': 'AUTH_2a8cbfbb8ad7411c8465f57311527937',
- # 'container_name': 'testcontainer2'}
- p = req.path_info
- if isinstance(p, unicode):
- p = p.encode('utf-8')
- except ValueError:
- self.logger.increment('errors')
- return HTTPNotFound(request=req)
- if not controller:
- self.logger.increment('errors')
- return HTTPPreconditionFailed(request=req, body='Bad URL')
- if self.deny_host_headers and req.host.split(':')[0] in self.deny_host_headers:
- return HTTPForbidden(request=req, body='Invalid host header')
- # server_type:不同的类定义了不同的server_type;
- # server_type = 'Object';
- # server_type = 'Account';
- # server_type = 'Base';
- # server_type = 'Container';
- self.logger.set_statsd_prefix('proxy-server.' + controller.server_type.lower())
- # 获取控制器类的实例化对象;
- controller = controller(self, **path_parts)
- if 'swift.trans_id' not in req.environ:
- # if this wasn't set by an earlier middleware, set it now
- trans_id = generate_trans_id(self.trans_id_suffix)
- req.environ['swift.trans_id'] = trans_id
- self.logger.txn_id = trans_id
- req.headers['x-trans-id'] = req.environ['swift.trans_id']
- controller.trans_id = req.environ['swift.trans_id']
- self.logger.client_ip = get_remote_client(req)
- try:
- # 执行具体控制器类中的指定方法;
- handler = getattr(controller, req.method)
- # req.method = DELETE
- # controller = <swift.proxy.controllers.obj.ObjectController object at 0x2acc110>
- # handler = <bound method ObjectController.DELETE of <swift.proxy.controllers.obj.ObjectController object at 0x2acc110>>
- getattr(handler, 'publicly_accessible')
- except AttributeError:
- allowed_methods = getattr(controller, 'allowed_methods', set())
- return HTTPMethodNotAllowed(request=req, headers={'Allow': ', '.join(allowed_methods)})
- if 'swift.authorize' in req.environ:
- resp = req.environ['swift.authorize'](req)
- if not resp:
- del req.environ['swift.authorize']
- else:
- if not getattr(handler, 'delay_denial', None):
- return resp
- req.environ['swift.orig_req_method'] = req.method
- # handler = <bound method ObjectController.DELETE of <swift.proxy.controllers.obj.ObjectController object at 0x2acc110>>
- return handler(req)
- except HTTPException as error_response:
- return error_response
- except (Exception, Timeout):
- self.logger.exception(_('ERROR Unhandled exception in request'))
- return HTTPServerError(request=req)
根据给定的HTTP的请求路径path获取控制器(ObjectController、ContainerController、AccountController)来处理请求;
2.controller = controller(self, **path_parts)
获取控制器类的实例化对象;
3.handler = getattr(controller, req.method)
执行具体控制器类中的指定方法;
4.return handler(req)
调用相应控制器中具体的方法来对req进行处理;
接下来将要调用的就是/swift/proxy/controllers/account.py或/swift/proxy/controllers/container.py或/swift/proxy/controllers/obj.py下面的PUT,POST,DELETE,GET,HEAD等方法;然后再在具体的方法中实现到具体存储服务(Object-server、Container-server、Account-server)的连接,继而调用其下具体的PUT,POST,DELETE,GET,HEAD等方法来进行请求req的实现;
该请求的转发过程是这样的:
首先根据请求组成一个字符串 PREFIX+account+container+object+POPFREIX进行MD5值的运算,然后根据计算结果进行移位获得key,而该key值就是我们需要的虚节点的数值,然后根据获得的虚节点的值,进行对应表的查找,获得虚节点和节点的对应值,然后依次在这些节点中进行数据的查找,如果找到了,则重新新建请求向指定的存储节点发送请求,进行分发,最后将返回的内容返回给客户端即可