【Nginx】反向代理流程二

0.概述

接受上游的HTTP包体：
(1)proxy_buffers
(2)proxy_buffering 
(3)proxy_max_temp_file_size
(4)proxy_temp_path


及时转发包体：
(1)proxy_busy_buffers_size


接受上游时速度相关指令：
(1)proxy_read_timeout
(2)proxy_limit_rate


上游包体持久化：
(1)proxy_store_access
(2)proxy_store

1.http反向代理流程

接受响应头部--由Nginx的框架upstream来处理的
处理响应头部--由各个反向代理模块来处理的,比如说gipc,fastcgi,uwsgi,httproxy.会有一个
              最大的内存长度来限制,就是通过proxy_buffer_size来限制.


上游包体的持久化

2.proxy_buffer_size

Syntax:	proxy_buffer_size size;
Default:proxy_buffer_size 4k|8k;
Context:http, server, location
Sets the size of the buffer used for reading the first part of the response received from 
the proxied server. This part usually contains a small response header. By default, the 
buffer size is equal to one memory page. This is either 4K or 8K, depending on a platform.
It can be made smaller, however.

proxy_buffer_size限定了接收的上游的http的response中的header的最大值.
所以当上游的server发送的http响应如果有set cookie这种特别长的header,
可能就会导致整个全部response的header超出这个值,如果超过这个值,这个请求
就不能被nginx正确地处理.
如果超长了,在errorlog中会有一条信息:upstream sent too big header.

3.proxy_buffering

Syntax:	proxy_buffering on | off;
Default:	
proxy_buffering on;
Context:	http, server, location
Enables or disables buffering of responses from the proxied server.

When buffering is enabled, nginx receives a response from the proxied server as soon as 
possible, saving it into the buffers set by the proxy_buffer_size and proxy_buffers 
directives. If the whole response does not fit into memory, a part of it can be saved to 
a temporary file on the disk. Writing to temporary files is controlled by the 
proxy_max_temp_file_size and proxy_temp_file_write_size directives.

When buffering is disabled, the response is passed to a client synchronously, immediately 
as it is received. nginx will not try to read the whole response from the proxied server. 
The maximum size of the data that nginx can receive from the server at a time is set by 
the proxy_buffer_size directive.

Buffering can also be enabled or disabled by passing “yes” or “no” in the “X-Accel-
Buffering” response header field. This capability can be disabled using the 
proxy_ignore_headers directive.

这个命令决定着当nginx收到上游服务器的响应的时候是等待接受完整之后一次性发送响应头部
还是一边接受一边发送,
proxy_buffering on      接收到完整的响应包体一起发送响应头部 
proxy_buffering off     一边接受一边发送

这两种方式各有各的好处,
通常情况下我们是设置成proxy_buffering on的,我们认为上游服务和nginx之间走的是企业的内网，
网速更快,如果边接收边发的话,由于客户端与Nginx之间的网速可能会较慢,会导致比较大的http的
响应的body的时候nginx长时间与上游建立连接,但一般上游服务器如jungle或者tomcat等并发能力
相对很弱并不适合高并发的连接.
当然假如我们此处用的是proxy_buffering off的话,能让客户端及时接收到响应,特别是对于body比
较大的情况下,不用客户端再去等待.

特定header-----X-Accel-Buffering,只有Nginx才会认这个header,当上游服务器在response中加入
这个header,如果配置成yes，就会强制要求nginx要接收完上游的http的body，再向客户端发送.

4.proxy_buffers

Syntax:	proxy_buffers number size;
Default:	
proxy_buffers 8 4k|8k;
Context:	http, server, location
Sets the number and size of the buffers used for reading a response from the proxied 
server, for a single connection. By default, the buffer size is equal to one memory page.
This is either 4K or 8K, depending on a platform.


当接收上游服务的包体的时候,即使设置了proxy_buffering on(接收完一起发送响应头部),也不一定
会向磁盘中写入包体,因为如果包体很小的话,在内存中就可以放下,此时就没有必要再去磁盘中.
提供proxy_buffers number size;就是设置不用将包体写入磁盘的最大内存大小.

5.proxy_max_temp_file_size

Syntax:	proxy_max_temp_file_size size;
Default:	
proxy_max_temp_file_size 1024m;
Context:	http, server, location
When buffering of responses from the proxied server is enabled, and the whole response does 
not fit into the buffers set by the proxy_buffer_size and proxy_buffers directives, a part 
of the response can be saved to a temporary file. This directive sets the maximum size of 
the temporary file. The size of data written to the temporary file at a time is set by the
 proxy_temp_file_write_size directive.

The zero value disables buffering of responses to temporary files.

This restriction does not apply to responses that will be cached or stored on disk.

限制写入磁盘中的文件的最大值.如果上游返回一个超出配置的大小的文件,会出错.

6.proxy_temp_file_write_size

Syntax:	proxy_temp_file_write_size size;
Default:	
proxy_temp_file_write_size 8k|16k;
Context:	http, server, location
Limits the size of data written to a temporary file at a time, when buffering of responses
from the proxied server to temporary files is enabled. By default, size is limited by two 
buffers set by the proxy_buffer_size and proxy_buffers directives. The maximum size of a 
temporary file is set by the proxy_max_temp_file_size directive.

每一次向临时磁盘文件中写入的字节数.

 7.proxy_temp_path

Syntax:	proxy_temp_path path [level1 [level2 [level3]]];
Default:	
proxy_temp_path proxy_temp;
Context:	http, server, location
Defines a directory for storing temporary files with data received from proxied servers. 
Up to three-level subdirectory hierarchy can be used underneath the specified directory. 
For example, in the following configuration

proxy_temp_path /spool/nginx/proxy_temp 1 2;
a temporary file might look like this:

/spool/nginx/proxy_temp/7/45/00000123457
See also the use_temp_path parameter of the proxy_cache_path directive.


设定了临时文件的存放目录以及使用几级目录.

8. proxy_busy_buffers_size

Syntax:	proxy_busy_buffers_size size;
Default:	
proxy_busy_buffers_size 8k|16k;
Context:	http, server, location
When buffering of responses from the proxied server is enabled, limits the total size of 
buffers that can be busy sending a response to the client while the response is not yet 
fully read. In the meantime, the rest of the buffers can be used for reading the response 
and, if needed, buffering part of the response to a temporary file. By default, size is 
limited by the size of two buffers set by the proxy_buffer_size and proxy_buffers 
directives.

虽然我们缓存了所有的响应，但我们希望更加及时地向客户端发送部分响应.
比如说我们收到1G的文件,当我们接收到前8k或者16k的时候就想要先向客户
端转发这部分响应的话,就可以使用这个命令.

9.proxy_read_timeout

Syntax:	proxy_read_timeout time;
Default:proxy_read_timeout 60s;
Context:http, server, location
Defines a timeout for reading a response from the proxied server. The timeout is set only 
between two successive read operations, not for the transmission of the whole response. If 
the proxied server does not transmit anything within this time, the connection is closed.

tcp层两次读取之间最大间隔时间,如果超过这个时间，连接会被关闭.

10. proxy_limit_rate

Syntax:	proxy_limit_rate rate;
Default:	
proxy_limit_rate 0;
Context:	http, server, location
This directive appeared in version 1.7.7.

Limits the speed of reading the response from the proxied server. The rate is specified in 
bytes per second. The zero value disables rate limiting. The limit is set per a request, 
and so if nginx simultaneously opens two connections to the proxied server, the overall 
rate will be twice as much as the specified limit. The limitation works only if buffering 
of responses from the proxied server is enabled.

限制了读取上游服务器响应的速度.
当其被设置成0的时候表示不限制读取上游服务器响应的速度.

11.proxy_store

Syntax:	proxy_store on | off | string;
Default:proxy_store off;
Context:http, server, location
Enables saving of files to a disk. The on parameter saves files with paths corresponding
 to the directives alias or root. The off parameter disables saving of files. In 
addition, the file name can be set explicitly using the string with variables:

proxy_store /data/www$original_uri;
The modification time of files is set according to the received “Last-Modified” response 
header field. The response is first written to a temporary file, and then the file is 
renamed. Starting from version 0.8.9, temporary files and the persistent store can be 
put on different file systems. However, be aware that in this case a file is copied 
across two file systems instead of the cheap renaming operation. It is thus recommended 
that for any given location both saved files and a directory holding temporary files, 
set by the proxy_temp_path directive, are put on the same file system.


This directive can be used to create local copies of static unchangeable files, e.g.:

location /images/ {
    root               /data/www;
    error_page         404 = /fetch$uri;
}

location /fetch/ {
    internal;

    proxy_pass         http://backend/;
    proxy_store        on;
    proxy_store_access user:rw group:rw all:r;
    proxy_temp_path    /data/temp;

    alias              /data/www/;
}
or like this:

location /images/ {
    root               /data/www;
    error_page         404 = @fetch;
}

location @fetch {
    internal;

    proxy_pass         http://backend;
    proxy_store        on;
    proxy_store_access user:rw group:rw all:r;
    proxy_temp_path    /data/temp;

    root               /data/www;
}


on      --改到root或者alias对应的目录下
off     --不开启
string  --以变量方式指定文件存放位置

12.proxy_store_access

Syntax:	proxy_store_access users:permissions ...;
Default:	
proxy_store_access user:rw;
Context:	http, server, location
Sets access permissions for newly created files and directories, e.g.:

proxy_store_access user:rw group:rw all:r;
If any group or all access permissions are specified then user permissions may be omitted:

proxy_store_access group:rw all:r;


上游发回来的响应也许是一个完整的文件,nginx应该将其存下来以便后续使用.
对临时文件的存储的权限的设置.

13.上游包持久化示例与说明

上游服务器返回的内容会在/tmp目录下持久化
测试文件：/usr/local/niginx/html/a.test
curl 127.0.0.1:8080/a.test
上游服务器成功以后会将a.test这个文件持久化到代理服务器指定的/tmp下.

======================================================================================
反向代理服务器的配置：
======================================================================================
worker_processes  4;  

events {
    worker_connections  1024;
}
http {

    upstream local{
        server 127.0.0.1:8081;
      }   

    server  {
        listen       8080;
        server_name localhost;
        root /tmp;
         location / { 
               proxy_pass http://local;
               proxy_store on; 
               proxy_store_access user:rw group:rw all:r; 
         }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

    }   
}





======================================================================================
上游服务器的配置
======================================================================================
worker_processes  8;  
worker_cpu_affinity 00000001 00000010 00000100 00001000 00010000 00100000 01000000 10000000;
events {
    worker_connections  1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;
    keepalive_timeout  65; 
    server {
        listen       8081;
        server_name  localhost;
        root html;
        location / { 
        }
        location /test{
             return 200 '8081 server response.
uri:$uri
method:$request_method
request:$request
http_name:$http_name
cur_time:$time_local
\n';

        }
        location /images/ {
                        root /usr/local/niginx/;
                }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

    }   

}