【Django的那些事】request、response全过程

前言：

上一篇文章初步分析了django runserver启动全过程，这篇文章会对服务启动后，如何处理用户请求以及响应进行分析。

序言：

django中用户的请求的url是如何一步步的跳转到视图函数（本篇文章主要针对FBV进行分析），又是如何加载django的中间件以及中间件的加载顺序，带着这些问题，我们依次进行分析。

一、django 监听用户请求

在上一篇文章中的最后，django会一直监听用户的请求。如下：

监听到用户请求后，会进行self.process_request(request, client_address)，通过这个函数来处理以及响应并结束本次请求过程。

二、django request、response全过程源码分析

接下来会根据下图，从上篇的一直监听用户请求到请求和响应结束，整个过程结合源码进行分析【django 版本2.1.1】

1、socketserver.py

def serve_forever(self, poll_interval=0.5):
        """Handle one request at a time until shutdown.

        Polls for shutdown every poll_interval seconds. Ignores
        self.timeout. If you need to do periodic tasks, do them in
        another thread.
        """
        self.__is_shut_down.clear()
        try:
            # XXX: Consider using another file descriptor or connecting to the
            # socket to wake this up instead of polling. Polling reduces our
            # responsiveness to a shutdown request and wastes cpu at all other
            # times.
            with _ServerSelector() as selector:
                selector.register(self, selectors.EVENT_READ)
				# 一直监听用户请求
                while not self.__shutdown_request:
                    ready = selector.select(poll_interval)
                    if ready:
                    	# 请求处理
                        self._handle_request_noblock()

                    self.service_actions()
        finally:
            self.__shutdown_request = False
            self.__is_shut_down.set()
            
 def _handle_request_noblock(self):
        """Handle one request, without blocking.

        I assume that selector.select() has returned that the socket is
        readable before this function was called, so there should be no risk of
        blocking in get_request().
        """
        try:
            request, client_address = self.get_request()
        except OSError:
            return
        if self.verify_request(request, client_address):
            try:
                self.process_request(request, client_address)
            except:
                self.handle_error(request, client_address)
                self.shutdown_request(request)
        else:
            self.shutdown_request(request)
            
def process_request(self, request, client_address):
    """Start a new thread to process the request."""
    t = threading.Thread(target = self.process_request_thread,
                         args = (request, client_address))
    t.daemon = self.daemon_threads
    t.start()

def process_request_thread(self, request, client_address):
    """Same as in BaseServer but as a thread.

    In addition, exception handling is done here.

    """
    try:
        self.finish_request(request, client_address)
        self.shutdown_request(request)
    except:
        self.handle_error(request, client_address)
        self.shutdown_request(request)

def finish_request(self, request, client_address):
    """Finish one request by instantiating RequestHandlerClass."""
    print("然后我又进去这里了...")
    self.RequestHandlerClass(request, client_address, self)

class BaseRequestHandler:
    def __init__(self, request, client_address, server):
        self.request = request
        self.client_address = client_address
        self.server = server
        self.setup()
        try:
        	# 处理request
            self.handle()
        finally:
        	# 结束
            self.finish()

1、serve_forever(self, poll_interval=0.5)函数主要是循环定时的监听用户请求，用户请求过来则会进入self._handle_request_noblock()函数进行request的处理以及响应，该函数时请求的入口函数。
2、 _handle_request_noblock(self)函数主要是处理用户请求并不阻塞，里面的self.process_request(request, client_address)函数时主要函数。
3、process_request(self, request, client_address)函数中_handle_request_noblock(self)调用的self.process_request(request, client_address)被class ThreadingMixIn重写【因为runserver启动过程中该函数继承 httpd_cls = type(‘WSGIServer’, (socketserver.ThreadingMixIn, server_cls), {})】。在该函数中会启动一个线程专门来处理用户本次的request请求。
4、process_request_thread(self, request, client_address)函数主要完成两个工作，第一就是调用self.finish_request(request, client_address)处理请求，第二个就是self.finish_request(request, client_address)结束请求。
5、finish_request(self, request, client_address)函数【httpd_cls继承了ThreadingMixIn和WSGIServer，WSGIServer继承了HTTPServer，HTTPServer继承了TCPServer，TCPServer继承了BaseServer，class ThreadingMixIn 没有finish_request方法使用的BaseServer的方法。
】，该函数实例化一个类，实例化哪个类，看runserver启动过程的时候，传入的一个handler【httpd = httpd_cls(server_address, WSGIRequestHandler, ipv6=ipv6)】，也就是WSGIRequestHandler,但是进入该类发现没有__init__或者__call__方法，看它的继承关系【继承WSGIRequestHandler，同样没有__init__方法，继续继承BaseHTTPRequestHandler，还是没有__init__方法,继续继承StreamRequestHandler，一直继承到BaseRequestHandler，在该类里面有__init__方法】。
6、**init**上面的函数实例化类后，进行类的初始化，然后在初始化里面调用了self.handle()和self.finish()，更具一步步的继承关系，找到在最开始的类class WSGIRequestHandler执行self.handel()方法。

2、django\core\servers\basehttp.py

class WSGIRequestHandler(simple_server.WSGIRequestHandler):
	def handle(self):
	       """Copy of WSGIRequestHandler.handle() but with different ServerHandler"""
	       self.raw_requestline = self.rfile.readline(65537)
	       if len(self.raw_requestline) > 65536:
	           self.requestline = ''
	           self.request_version = ''
	           self.command = ''
	           self.send_error(414)
	           return
	
	       if not self.parse_request():  # An error code has been sent, just exit
	           return
		   # 实例化对象
	       handler = ServerHandler(
	           self.rfile, self.wfile, self.get_stderr(), self.get_environ()
	       )
	       handler.request_handler = self      # backpointer for logging
	       handler.run(self.server.get_app())

1、该函数主要是实例化对象，然后调用对象的run方法。实例化的对象可以根据上面的方法一次次抽离，查看也就是class BaseHandler。在此主要看下这个传递的self.server.get_app()传递的app是谁【httpd.set_app(wsgi_handler)】这个就是传递app，这些都是在runserver启动的时候传递的，位于函数django\core\servers\basehttp.py run方法【该方法需要留意，上面的函数分析中继承的类也是在该函数中初始化的】，那我们看看这个app是什么【handler = self.get_handler(*args, **options) 位于django\core\management\commands\runserver.py inner_run方法内】，在看看get_handler()—>get_internal_wsgi_application()—>…return StaticFilesHandler(handler)【注意：这里为这个实例化队形传递了一个WSGIHandler()对象，下文会用到】静态文件handler，为何要使用这个下面会阐述。

3、wsgiref\handlers.py

class BaseHandler:
    def run(self, application):
        """Invoke the application"""
        # Note to self: don't move the close()!  Asynchronous servers shouldn't
        # call close() from finish_response(), so if you close() anywhere but
        # the double-error branch here, you'll break asynchronous servers by
        # prematurely closing.  Async servers must return from 'run()' without
        # closing if there might still be output to iterate over.
        try:
        	# 初始化环境
            self.setup_environ()
            # 处理请求以及开始response
            self.result = application(self.environ, self.start_response)
            self.finish_response()
        except:
            try:
                self.handle_error()
            except:
                # If we get an error handling an error, just give up already!
                self.close()
                raise   # ...and let the actual server figure it out.

该函数主要是调用实例化StaticFilesHandler，然后在其内部执行操作。
4、django\contrib\staticfiles\handlers.py

class StaticFilesHandler(WSGIHandler):
	def __init__(self, application):
        self.application = application
        self.base_url = urlparse(self.get_base_url())
        super().__init__()
	
	def __call__(self, environ, start_response):
        if not self._should_handle(get_path_info(environ)):
            return self.application(environ, start_response)
        return super().__call__(environ, start_response)

这里把这个类
1、__init__函数第一个是设置application，这个application就是上面提到WSGIHandler()对象；第二设置base_url，其实在runserver启动过程中就会加载所有的url路由。
2、__call__方法，该方法就是实例化之后就会调用的方法，主要看里面的return self.application(environ, start_response)这里面的application(environ, start_response)就是实例化的WSGIHandler()对象,所以此时也就是调用WSGIHandler()对象里面的__call__方法。

4、django\core\handlers\wsgi.py

class WSGIHandler(base.BaseHandler):
    request_class = WSGIRequest

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.load_middleware()

    def __call__(self, environ, start_response):
        set_script_prefix(get_script_name(environ))
        signals.request_started.send(sender=self.__class__, environ=environ)
        # 请求头以及获取请求url
        request = self.request_class(environ)
        # 响应
        response = self.get_response(request)

        response._handler_class = self.__class__

        status = '%d %s' % (response.status_code, response.reason_phrase)
        response_headers = list(response.items())
        for c in response.cookies.values():
            response_headers.append(('Set-Cookie', c.output(header='')))
        start_response(status, response_headers)
        if getattr(response, 'file_to_stream', None) is not None and environ.get('wsgi.file_wrapper'):
            response = environ['wsgi.file_wrapper'](response.file_to_stream)
        return response

1、__init__方法里面的self.load_middleware()加载中间件
2、__call__方法里面 request = self.request_class(environ)获取到用户请求的url后面就开始配置runserver启动时候加载的url； response = self.get_response(request)获取用户url对应的响应。

5、django\core\handlers\base.py

class BaseHandler:
	"各个中间件"
    _view_middleware = None
    _template_response_middleware = None
    _exception_middleware = None
    _middleware_chain = None

    def get_response(self, request):
        """Return an HttpResponse object for the given HttpRequest."""
        # Setup default url resolver for this thread
        set_urlconf(settings.ROOT_URLCONF)
		# 中间链
        response = self._middleware_chain(request)

        response._closable_objects.append(request)

        # If the exception handler returns a TemplateResponse that has not
        # been rendered, force it to be rendered.
        if not getattr(response, 'is_rendered', True) and callable(getattr(response, 'render', None)):
            response = response.render()

        if response.status_code >= 400:
            log_response(
                '%s: %s', response.reason_phrase, request.path,
                response=response,
                request=request,
            )

        return response
	def _get_response(self, request):
        """
        Resolve and call the view, then apply view, exception, and
        template_response middleware. This method is everything that happens
        inside the request/response middleware.
        """
        response = None

        if hasattr(request, 'urlconf'):
            urlconf = request.urlconf
            set_urlconf(urlconf)
            resolver = get_resolver(urlconf)
        else:
            resolver = get_resolver()
		# url配置view函数
        resolver_match = resolver.resolve(request.path_info)
        callback, callback_args, callback_kwargs = resolver_match
        request.resolver_match = resolver_match

        # Apply view middleware
        # 加载视图中间件
        for middleware_method in self._view_middleware:
            response = middleware_method(request, callback, callback_args, callback_kwargs)
            if response:
                break

        if response is None:
        	# 回调函数
            wrapped_callback = self.make_view_atomic(callback)
            try:
            	# 返回响应
                response = wrapped_callback(request, *callback_args, **callback_kwargs)
            except Exception as e:
                response = self.process_exception_by_middleware(e, request)

        # Complain if the view returned None (a common error).
        if response is None:
            if isinstance(callback, types.FunctionType):    # FBV
                view_name = callback.__name__
            else:                                           # CBV
                view_name = callback.__class__.__name__ + '.__call__'

            raise ValueError(
                "The view %s.%s didn't return an HttpResponse object. It "
                "returned None instead." % (callback.__module__, view_name)
            )

        # If the response supports deferred rendering, apply template
        # response middleware and then render the response
        elif hasattr(response, 'render') and callable(response.render):
            for middleware_method in self._template_response_middleware:
                response = middleware_method(request, response)
                # Complain if the template response middleware returned None (a common error).
                if response is None:
                    raise ValueError(
                        "%s.process_template_response didn't return an "
                        "HttpResponse object. It returned None instead."
                        % (middleware_method.__self__.__class__.__name__)
                    )

            try:
                response = response.render()
            except Exception as e:
                response = self.process_exception_by_middleware(e, request)

        return response
        
    def load_middleware(self):
        """
        Populate middleware lists from settings.MIDDLEWARE.

        Must be called after the environment is fixed (see __call__ in subclasses).
        """
        self._view_middleware = []
        self._template_response_middleware = []
        self._exception_middleware = []

        handler = convert_exception_to_response(self._get_response)
        for middleware_path in reversed(settings.MIDDLEWARE):
            middleware = import_string(middleware_path)
            try:
                mw_instance = middleware(handler)
            except MiddlewareNotUsed as exc:
                if settings.DEBUG:
                    if str(exc):
                        logger.debug('MiddlewareNotUsed(%r): %s', middleware_path, exc)
                    else:
                        logger.debug('MiddlewareNotUsed: %r', middleware_path)
                continue

            if mw_instance is None:
                raise ImproperlyConfigured(
                    'Middleware factory %s returned None.' % middleware_path
                )

            if hasattr(mw_instance, 'process_view'):
                self._view_middleware.insert(0, mw_instance.process_view)
            if hasattr(mw_instance, 'process_template_response'):
                self._template_response_middleware.append(mw_instance.process_template_response)
            if hasattr(mw_instance, 'process_exception'):
                self._exception_middleware.append(mw_instance.process_exception)

            handler = convert_exception_to_response(mw_instance)

        # We only assign to this when initialization is complete as it is used
        # as a flag for initialization being complete.
        self._middleware_chain = handler

#位于：django\core\handlers\exception.py
def convert_exception_to_response(get_response):
    """
    Wrap the given get_response callable in exception-to-response conversion.

    All exceptions will be converted. All known 4xx exceptions (Http404,
    PermissionDenied, MultiPartParserError, SuspiciousOperation) will be
    converted to the appropriate response, and all other exceptions will be
    converted to 500 responses.

    This decorator is automatically applied to all middleware to ensure that
    no middleware leaks an exception and that the next middleware in the stack
    can rely on getting a response instead of an exception.
    """
    @wraps(get_response)
    def inner(request):
        try:
            response = get_response(request)
        except Exception as exc:
            response = response_for_exception(request, exc)
        return response
    return inner

    

主要是加载settings.MIDDLEWARE也就是

1、主要看class MiddlewareMixin 里面的__call__方法【注意这是在request到view视图这个过程从上到下依次加载，后面会在view 返回后response再从下到上依次再加载，】，中间件加载不在此次谈论范围之内【后面的文件将会介绍django中间件】，可以借助下面一张图大致了解其加载过程：request中间件、view中间件、template中间件、exception中间件以及response中间件，

然后跳转到handler = convert_exception_to_response(self._get_response)函数中的self._get_response
2、self._get_response函数
（1）：resolver_match = resolver.resolve(request.path_info)url匹配到对应的view函数路径
（2）：wrapped_callback = self.make_view_atomic(callback) 回调视图函数
（3）：response = wrapped_callback(request, *callback_args, **callback_kwargs) 回调函数传参，并返回试图函数响应。
（4）：self._template_response_middleware 加载模板中间件

6、django\utils\deprecation.py

class MiddlewareMixin:
    def __init__(self, get_response=None):
        self.get_response = get_response
        super().__init__()

    def __call__(self, request):
        response = None
        # 加载resquest中间件
        if hasattr(self, 'process_request'):
            response = self.process_request(request)
        response = response or self.get_response(request)
        # 加载response中间件
        if hasattr(self, 'process_response'):
            response = self.process_response(request, response)
        return response

主要看__call__方法，里面里面首先调用加载resquest中间件，前面提到，然后用户视图函数响应后再加载response中间件。

7、response结束
从self.get_response(request)返回，然后添加用户状态[status:‘200 ok’]
跳转出来self.finish_response 用户响应到此也就结束了。

def finish_response(self):
    """Send any iterable data, then close self and the iterable

    Subclasses intended for use in asynchronous servers will
    want to redefine this method, such that it sets up callbacks
    in the event loop to iterate over the data, and to call
    'self.close()' once the response is finished.
    """
    try:
        if not self.result_is_file() or not self.sendfile():
            for data in self.result:
            	# 数据反馈给前端
                self.write(data)
            self.finish_content()
    finally:
        self.close()

其实看看result是什么，原来是渲染的h5页面。
然后： self.finish_response()—>self.finish()—>self.shutdown_request(request)


结束

三、总结

好了，从用户一条请求到服务器捕捉到请求，创建线程处理请求，再到一步步加载中间件、匹配路由、加载视图中间件、视图函数响应、加载template中间件、渲染、response中间件到结束用户response等过程都进行了分析。本文也就告一段落，后面的文章会分析django rest framework框架的内容以及基于该框架的CBV视图。

注意：文中的self.handler()以及application等过程的获取要一层层的查找继承，这点需要耐心查找、匹配。
本文也是通过postman模拟百十次请求然后断点调试才慢慢的找到其整个流程，如果有不对的地方，希望可以提出来，共同进步。