1.查看当前系统中所有TCP的实时运行状态:
netstat -n | awk '/^tcp/ {++S[$NF]} END {for(a in S) print a, S[a]}'
LAST_ACK 23
SYN_RECV 49
CLOSE_WAIT 6
ESTABLISHED 103195
FIN_WAIT1 69
FIN_WAIT2 928
TIME_WAIT 20045
2.TIME_WAIT 数很明显不正常,经从网上查询资料,找到解决方案,需要修改nginx配置文件,修要修改两点:
①upstream中要加入keepalive,表示保活的TCP连接数,如下所示
upstream backend {
keepalive 500;
#A服务器
server 192.168.1.2:18234 weight=3;
#B服务器
server 192.168.1.3:18234 weight=3;
check interval=3000 rise=2 fall=5 timeout=2000 type=http;
}
②在每一个location下加入proxy_http_version 1.1; 和 proxy_set_header Connection ""; 如下图所示:
location /testService/testManage/ {
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Scheme $scheme; #向下传递https
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://backend/testService/testManage/;
}
3.经过实际测试发现,在proxy_pass中使用upstream的,TIME_WAIT数量确实减少了,但是不使用upstream依然不会减少,所以在用nginx做代理时,一定要用upstream,即使只转向一个tomcat,也要这么做。
4.使用netstat -n | awk '/^tcp/ {++S[$NF]} END {for(a in S) print a, S[a]}',如下图所示,依然有三千多的TIME_WAIT,经分析发现是nginx做静态代理的请求,即使加入proxy_http_version 1.1; 和 proxy_set_header Connection ""; 依然没有效果,暂时没有找到好的解决方法
LAST_ACK 30
SYN_RECV 52
CLOSE_WAIT 11
ESTABLISHED 101595
FIN_WAIT1 109
FIN_WAIT2 674
TIME_WAIT 3489
5.修改Linux的系统变量,但是感觉用处不大,输入sysctl -p,可以查看当前系统参数
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
vm.swappiness = 0
net.ipv4.neigh.default.gc_stale_time = 120
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_announce = 2
net.ipv4.tcp_max_tw_buckets = 100000
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_fin_timeout = 10
net.ipv4.ip_local_port_range = 25000 61000
net.ipv4.tcp_timestamps = 0
net.core.somaxconn = 2048
vm.overcommit_memory = 1
net.nf_conntrack_max = 400000
net.netfilter.nf_conntrack_tcp_timeout_established = 10800
kernel.sysrq = 1
net.ipv4.tcp_mem = 1894275 1010280 3030840
net.ipv4.tcp_wmem = 4096 4096 6291456
net.ipv4.tcp_rmem = 4096 4096 6291456
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_orphan_retries = 3
net.ipv4.tcp_keepalive_probes = 5
net.core.netdev_max_backlog = 3000