nginx：upstream的keep-alive是怎么实现的

数据结构

//变量kcf
typedef struct {
    ngx_uint_t                         max_cached;  //keepalive参数，有多少ngx_http_upstream_keepalive_cache_t
    ngx_uint_t                         requests;    //每个连接处理多少请求
    ngx_msec_t                         timeout;     //空闲超时

    ngx_queue_t                        cache;       //ngx_http_upstream_keepalive_cache_t链表,有connection
    ngx_queue_t                        free;        //ngx_http_upstream_keepalive_cache_t链表,空闲的

    ngx_http_upstream_init_pt          original_init_upstream;
    ngx_http_upstream_init_peer_pt     original_init_peer;

} ngx_http_upstream_keepalive_srv_conf_t;


//变量kp，connection cache，串在free/cache上
typedef struct {
    ngx_http_upstream_keepalive_srv_conf_t  *conf;

    ngx_queue_t                        queue; //串在cache/free上
    ngx_connection_t                  *connection;

    socklen_t                          socklen;
    ngx_sockaddr_t                     sockaddr;

} ngx_http_upstream_keepalive_cache_t;


//request选择server的方法，串在负载均衡算法之前
typedef struct {
    ngx_http_upstream_keepalive_srv_conf_t  *conf;

    ngx_http_upstream_t               *upstream;

    void                              *data;

    ngx_event_get_peer_pt              original_get_peer;
    ngx_event_free_peer_pt             original_free_peer;

#if (NGX_HTTP_SSL)
    ngx_event_set_peer_session_pt      original_set_session;
    ngx_event_save_peer_session_pt     original_save_session;
#endif

} ngx_http_upstream_keepalive_peer_data_t;

算法

关键的两个函数

ngx_http_upstream_get_keepalive_peer()

遍历ngx_http_upstream_keepalive_srv_conf_t.cache链表，通过比较地址，选出对应的connection

 

ngx_http_upstream_free_keepalive_peer()

把connection归还到ngx_http_upstream_keepalive_srv_conf_t.free中

关闭

ngx_http_upstream_free_keepalive_peer（）
{
...
    //设置超时   
    ngx_add_timer(c->read, kp->conf->timeout);

    if (c->write->timer_set) {
        ngx_del_timer(c->write);
    }

    //空操作
    c->write->handler = ngx_http_upstream_keepalive_dummy_handler;
    //如果有错误、数据，就关闭连接
    c->read->handler = ngx_http_upstream_keepalive_close_handler;
...
}

总结

nginx的把所有空闲连接，放在一个链表中，需要时，从链表中遍历查找，这是O(n)算法，当后端连接数比较多时、空闲连接数很多时，需要优化为O(1)算法。