详解proxy_pass、upstream与resolver
应用场景
这里列举几个应用场景，下文会针对这几个场景并结合代码进行分析。
- proxy_pass + upstream
upstream foo.example.com {
server 127.0.0.1:8001; } server { listen 80; server_name localhost; location /foo { proxy_pass http://foo.example.com; } } 访问http://localhost/foo，proxy模块会将请求转发到127.0.0.1的8001端口上。
- 只有proxy_pass，没有upstream与resolver
server {
listen 80;
server_name localhost;
location /foo {
proxy_pass http://foo.example.com;
}
}
实际上是隐式创建了upstream，upstream名字就是foo.example.com。upstream模块利用本机设置的DNS服务器（或/etc/hosts），将foo.example.com解析成IP，访问http://localhost/foo，proxy模块会将请求转发到解析后的IP上。
如果本机未设置DNS服务器，或者DNS服务器无法解析域名，则nginx启动时会报类似如下错误：
nginx: [emerg] host not found in upstream "foo.example.com" in /path/nginx/conf/nginx.conf:110
- proxy_pass + resolver（变量设置域名）
server {
listen 80;
server_name localhost;
resolver 114.114.114.114;
location /foo {
set $foo foo.example.com;
proxy_pass http://$foo;
}
}
访问http://localhost/foo，nginx会动态利用resolver设置的DNS服务器（本机设置的DNS服务器或/etc/hosts无效），将域名解析成IP，proxy模块会将请求转发到解析后的IP上。
- proxy_pass + upstream（显式） + resolver（变量设置域名）
upstream foo.example.com {
server 127.0.0.1:8001;
}
server {
listen 80;
server_name localhost;
resolver 114.114.114.114;
location /foo {
set $foo foo.example.com;
proxy_pass http://$foo;
}
}
访问http://localhost/foo时，upstream模块会优先查找是否有定义upstream后端服务器，如果有定义则直接利用，不再走DNS解析。所以proxy模块会将请求转发到127.0.0.1的8001端口上。
- proxy_pass + upstream（隐式） + resolver（变量设置域名）
server {
listen 80;
server_name localhost;
resolver 114.114.114.114;
location /foo {
set $foo foo.example.com;
proxy_pass http://$foo;
}
location /foo2 {
proxy_pass http://foo.example.com;
}
}
location /foo2实际上是隐式定义了upstream foo.example.com，并由本地DNS服务器进行了域名解析，访问http://localhost/foo时，upstream模块会优先查找upstream，即隐式定义的foo.example.com，proxy模块会将请求转发到解析后的IP上。
- proxy_pass + resolver（不用变量设置域名）
server {
listen 80;
server_name localhost;
resolver 114.114.114.114; location /foo { proxy_pass http://foo.example.com; } } 不使用变量设置域名，则resolver的设置不起作用，此时相当于场景2，只有proxy_pass的场景。
- proxy_pass + upstream + resolver（不用变量设置域名）
upstream foo.example.com {
server 127.0.0.1:8001; } server { listen 80; server_name localhost; resolver 114.114.114.114; location /foo { proxy_pass http://foo.example.com; } } 不使用变量设置域名，则resolver的设置不起作用，此时相当于场景1 proxy_pass + upstream。
- proxy_pass 直接指定IP加端口号
server {
listen 80;
server_name localhost;
location /foo {
proxy_pass http://127.0.0.1:8001/;
}
}
实际上是隐式创建了upstream，proxy_pass会将请求转发到127.0.0.1的8001端口上。
主要代码
解析proxy_pass指令的代码：
static char *
ngx_http_proxy_pass(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
{
ngx_http_proxy_loc_conf_t *plcf = conf;
size_t add;
u_short port;
ngx_str_t *value, *url;
ngx_url_t u;
ngx_uint_t n;
ngx_http_core_loc_conf_t *clcf;
ngx_http_script_compile_t sc;
if (plcf->upstream.upstream || plcf->proxy_lengths) {
return "is duplicate"; } clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module); clcf->handler = ngx_http_proxy_handler; if (clcf->name.data[clcf->name.len - 1] == '/') { clcf->auto_redirect = 1; } value = cf->args->elts; url = &value[1]; /* 查找指令中$符号的位置，判断是否使用了变量 */ n = ngx_http_script_variables_count(url); if (n) { /* 使用变量设置域名 */ ngx_memzero(&sc, sizeof(ngx_http_script_compile_t)); sc.cf = cf; sc.source = url; sc.lengths = &plcf->proxy_lengths; sc.values = &plcf->proxy_values; sc.variables = n; sc.complete_lengths = 1; sc.complete_values = 1; if (ngx_http_script_compile(&sc) != NGX_OK) { return NGX_CONF_ERROR; } #if (NGX_HTTP_SSL) plcf->ssl = 1; #endif return NGX_CONF_OK; } if (ngx_strncasecmp(url->data, (u_char *) "http://", 7) == 0) { add = 7; port = 80; } else if (ngx_strncasecmp(url->data, (u_char *) "https://", 8) == 0) { #if (NGX_HTTP_SSL) plcf->ssl = 1; add = 8; port = 443; #else ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "https protocol requires SSL support"); return NGX_CONF_ERROR; #endif } else { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid URL prefix"); return NGX_CONF_ERROR; } ngx_memzero(&u, sizeof(ngx_url_t)); u.url.len = url->len - add; u.url.data = url->data + add; u.default_port = port; u.uri_part = 1; u.no_resolve = 1; plcf->upstream.upstream = ngx_http_upstream_add(cf, &u, 0); if (plcf->upstream.upstream == NULL) { return NGX_CONF_ERROR; } plcf->vars.schema.len = add; plcf->vars.schema.data = url->data; plcf->vars.key_start = plcf->vars.schema; ngx_http_proxy_set_vars(&u, &plcf->vars); plcf->location = clcf->name; if (clcf->named #if (NGX_PCRE) || clcf->regex #endif || clcf->noname) { if (plcf->vars.uri.len) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "\"proxy_pass\" cannot have URI part in " "location given by regular expression, " "or inside named location, " "or inside \"if\" statement, " "or inside \"limit_except\" block"); return NGX_CONF_ERROR; } plcf->location.len = 0; } plcf->url = *url; return NGX_CONF_OK; } upstream定义的后端服务器的处理逻辑，包括显式定义的和隐式定义的。隐式定义，即proxy_pass指定的后端服务器的地址没有显式用upstream定义，nginx内部会定义一个。
ngx_int_t
ngx_http_upstream_init_round_robin(ngx_conf_t *cf,
ngx_http_upstream_srv_conf_t *us)
{
ngx_url_t u;
ngx_uint_t i, j, n, w;
ngx_http_upstream_server_t *server;
ngx_http_upstream_rr_peer_t *peer, **peerp;
ngx_http_upstream_rr_peers_t *peers, *backup;
us->peer.init = ngx_http_upstream_init_round_robin_peer;
/*
* 使用upstream指令显式定义upstream
* 或者proxy_pass直接指定IP的场景
*/
if (us->servers) {
server = us->servers->elts;
n = 0;
w = 0; for (i = 0; i < us->servers->nelts; i++) { if (server[i].backup) { continue; } n += server[i].naddrs; w += server[i].naddrs * server[i].weight; } if (n == 0) { ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "no servers in upstream \"%V\" in %s:%ui", &us->host, us->file_name, us->line); return NGX_ERROR; } peers = ngx_pcalloc(cf->pool, sizeof(ngx_http_upstream_rr_peers_t)); if (peers == NULL) { return NGX_ERROR; } peer = ngx_pcalloc(cf->pool, sizeof(ngx_http_upstream_rr_peer_t) * n); if (peer == NULL) { return NGX_ERROR; } peers->single = (n == 1); peers->number = n; peers->weighted = (w != n); peers->total_weight = w; peers->name = &us->host; n = 0; peerp = &peers->peer; for (i = 0; i < us->servers->nelts; i++) { /* 设置sockaddr、weight、max_fails、fail_timeout等属性 */ } us->peer.data = peers; /* 处理backup servers相关逻辑 */ return NGX_OK; } /* 未使用upstream指令，使用proxy_pass隐式定义upstream */ /* an upstream implicitly defined by proxy_pass, etc. */ if (us->port == 0) { ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "no port in upstream \"%V\" in %s:%ui", &us->host, us->file_name, us->line); return NGX_ERROR; } ngx_memzero(&u, sizeof(ngx_url_t)); u.host = us->host; u.port = us->port; if (ngx_inet_resolve_host(cf->pool, &u) != NGX_OK) { if (u.err) { ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "%s in upstream \"%V\" in %s:%ui", u.err, &us->host, us->file_name, us->line); } return NGX_ERROR; } n = u.naddrs; peers = ngx_pcalloc(cf->pool, sizeof(ngx_http_upstream_rr_peers_t)); if (peers == NULL) { return NGX_ERROR; } peer = ngx_pcalloc(cf->pool, sizeof(ngx_http_upstream_rr_peer_t) * n); if (peer == NULL) { return NGX_ERROR; } peers->single = (n == 1); peers->number = n; peers->weighted = 0; peers->total_weight = n; peers->name = &us->host; peerp = &peers->peer; for (i = 0; i < u.naddrs; i++) { /* 设置sockaddr、weight、max_fails、fail_timeout等属性 */ } us->peer.data = peers; /* implicitly defined upstream has no backup servers */ return NGX_OK; } upstream模块初始化请求时的逻辑：
static void
ngx_http_upstream_init_request(ngx_http_request_t *r)
{
ngx_str_t *host;
ngx_uint_t i;
ngx_resolver_ctx_t *ctx, temp;
ngx_http_cleanup_t *cln;
ngx_http_upstream_t *u;
ngx_http_core_loc_conf_t *clcf;
ngx_http_upstream_srv_conf_t *uscf, **uscfp;
ngx_http_upstream_main_conf_t *umcf;
if (r->aio) {
return;
}
u = r->upstream;
/* NGX_HTTP_CACHE 等其他处理 */ cln->handler = ngx_http_upstream_cleanup; cln->data = r; u->cleanup = &cln->handler; if (u->resolved == NULL) { /* 如果没有使用resolver设置DNS，直接取upstream的设置 */ uscf = u->conf->upstream; } else { #if (NGX_HTTP_SSL) u->ssl_name = u->resolved->host; #endif host = &u->resolved->host; if (u->resolved->sockaddr) { if (u->resolved->port == 0 && u->resolved->sockaddr->sa_family != AF_UNIX) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "no port in upstream \"%V\"", host); ngx_http_upstream_finalize_request(r, u, NGX_HTTP_INTERNAL_SERVER_ERROR); return; } if (ngx_http_upstream_create_round_robin_peer(r, u->resolved) != NGX_OK) { ngx_http_upstream_finalize_request(r, u, NGX_HTTP_INTERNAL_SERVER_ERROR); return; } ngx_http_upstream_connect(r, u); return; } umcf = ngx_http_get_module_main_conf(r, ngx_http_upstream_module); uscfp = umcf->upstreams.elts; /* 在显式/隐式定义的upstream中查找 */ for (i = 0; i < umcf->upstreams.nelts; i++) { uscf = uscfp[i]; if (uscf->host.len == host->len && ((uscf->port == 0 && u->resolved->no_port) || uscf->port == u->resolved->port) && ngx_strncasecmp(uscf->host.data, host->data, host->len) == 0) { goto found; } } if (u->resolved->port == 0) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "no port in upstream \"%V\"", host); ngx_http_upstream_finalize_request(r, u, NGX_HTTP_INTERNAL_SERVER_ERROR); return; } temp.name = *host; ctx = ngx_resolve_start(clcf->resolver, &temp); if (ctx == NULL) { ngx_http_upstream_finalize_request(r, u, NGX_HTTP_INTERNAL_SERVER_ERROR); return; } if (ctx == NGX_NO_RESOLVER) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "no resolver defined to resolve %V", host); ngx_http_upstream_finalize_request(r, u, NGX_HTTP_BAD_GATEWAY); return; } ctx->name = *host; ctx->handler = ngx_http_upstream_resolve_handler; ctx->data = r; ctx->timeout = clcf->resolver_timeout; u->resolved->ctx = ctx; if (ngx_resolve_name(ctx) != NGX_OK) { u->resolved->ctx = NULL; ngx_http_upstream_finalize_request(r, u, NGX_HTTP_INTERNAL_SERVER_ERROR); return; } return; } found: if (uscf == NULL) { ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0, "no upstream configuration"); ngx_http_upstream_finalize_request(r, u, NGX_HTTP_INTERNAL_SERVER_ERROR); return; } #if (NGX_HTTP_SSL) u->ssl_name = uscf->host; #endif if (uscf->peer.init(r, uscf) != NGX_OK) { ngx_http_upstream_finalize_request(r, u, NGX_HTTP_INTERNAL_SERVER_ERROR); return; } u->peer.start_time = ngx_current_msec; if (u->conf->next_upstream_tries && u->peer.tries > u->conf->next_upstream_tries) { u->peer.tries = u->conf->next_upstream_tries; } ngx_http_upstream_connect(r, u); } 详细分析
场景1
解析proxy_pass的函数ngx_http_proxy_pass中，没有找到$符号（即，变量设置域名），走ngx_http_proxy_pass后半部分的处理逻辑。ngx_http_upstream_init_round_robin初始化upstream时，走显式定义upstream的逻辑。proxy_pass转发请求初始化时，ngx_http_upstream_init_request中直接使用upstream中的后端server建立连接。
场景2
ngx_http_upstream_init_round_robin初始化upstream时，走隐式定义upstream的逻辑，会调用ngx_inet_resolve_host对proxy_pass中的域名进行解析，设置upstream。proxy_pass转发请求初始化时，ngx_http_upstream_init_request中直接使用upstream中的设置，也就是利用本地设置的DNS服务器解析出的IP，建立连接。
场景3
解析proxy_pass指令时，找到了$符号，设置ngx_http_script_compile_t，并利用ngx_http_script_compile进行编译，不走后半部分逻辑。配置文件没有显式/隐式定义upstream，所以不会调用ngx_http_upstream_init_round_robin方法。proxy_pass转发请求初始化时，ngx_http_upstream_init_request中发现没有显式也没有隐式定义的upstream，随后调用ngx_resolve_start，对域名进行解析，之后将请求转发过去。
场景4
解析proxy_pass指令时，找到了$符号，设置ngx_http_script_compile_t，并利用ngx_http_script_compile进行编译，不走后半部分逻辑。显式调用了upstream，所以调用ngx_http_upstream_init_round_robin方法中的显式upstream的处理逻辑。proxy_pass转发请求初始化时，ngx_http_upstream_init_request中优先查找upstream，如果找到了，直接将请求转发到upstream中的后端server上。如果upstream中没有找到，则对域名进行解析，然后将请求转发到解析后的IP上。
场景5
基本与场景4相同，不同之处在于调用ngx_http_upstream_init_round_robin方法时，走隐式upstream部分的处理逻辑。
场景6
与场景2相同。
场景7
与场景1相同。
场景8
实际上是隐式创建了upstream，但是因为proxy_pass中指定了IP和端口号，所以ngx_http_upstream_init_round_robin初始化upstream时，us->servers不为空，所以走该函数的上半部分逻辑。与场景1有些类似。
https://www.jianshu.com/p/5caa48664da5
322
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
