关于net.core.somaxconn参数 【转】

关于net.core.somaxconn参数

 

 

我们线上服务器net.core.somaxconn都是默认的128,这个参数会影响到所有AF_INET类型socket的listen队列

Man 2 listen可以知道:

int listen(int s, int backlog);

The backlog parameter defines the maximum length the queue of pending connections may grow to. If a connection request

arrives with the queue full the client may receive an error with an indication of ECONNREFUSED or, if the underlying pro-

tocol supports retransmission, the request may be ignored so that retries succeed.

 

BUGS

If the socket is of type AF_INET, and the backlog argument is greater than the constant SOMAXCONN (128 in Linux 2.0 &

2.2), it is silently truncated to SOMAXCONN.

 

也就是说,web应用中listen函数的backlog会给我们内核参数的net.core.somaxconn 限制到128,在高突发的请求中可能会导致链接超时或者触发重传

 

比如nginx 定义NGX_LISTEN_BACKLOG默认到511, 却由于我们参数未曾优化会限制到128,只有128个connections can be queued in kernel listen
queue(by Igor Sysoev).

#define NGX_LISTEN_BACKLOG 511/ls.backlog = NGX_LISTEN_BACKLOG;/ if (listen(s, ls.backlog) == -1) {

 

相信其他应用比如squid也会有类似问题,突发的大并发connect请求会由于内核listen队列的限制导致链接超时或者重传,从而影响用户体验

 

 

以下是实验测试情况,使用2台机器分别以1000个并发,benchmark方式,请求服务器 ( 相当于2000个并发请求同时请求服务器 )

情景1,默认配置, net.core.somaxconn=128,服务器为nginx

测试客户端A:

Transactions: 2072870 hits

Availability: 99.99 %

Elapsed time: 179.59 secs

Data transferred: 6096.59 MB

Response time: 0.08 secs

Transaction rate: 11542.24 trans/sec

Throughput: 33.95 MB/sec

Concurrency: 927.34

Successful transactions: 2072871

Failed transactions: 300

Longest transaction: 45.30

Shortest transaction: 0.00

错误率大概是1.5%

 

测试客户端B:

Transactions: 1859454 hits

Availability: 99.99 %

Elapsed time: 179.11 secs

Data transferred: 5468.90 MB

Response time: 0.09 secs

Transaction rate: 10381.63 trans/sec

Throughput: 30.53 MB/sec

Concurrency: 904.45

Successful transactions: 1859454

Failed transactions: 276

Longest transaction: 49.60

Shortest transaction: 0.00

错误率大概也是1.5%

错误提示大都为:

socket: connection timed out

warning: socket: -1803417280 select timed out: Connection timed out

 

情景2,调整配置, net.core.somaxconn=8192, nginx显式配置 listen 80 default backlog=8192;

测试客户端A:

 

** SIEGE 2.69

** Preparing 1000 concurrent users for battle.

The server is now under siege...

Lifting the server siege... done.

Transactions: 1789818 hits

Availability: 100.00 %

Elapsed time: 180.00 secs

Data transferred: 5264.09 MB

Response time: 0.10 secs

Transaction rate: 9943.43 trans/sec

Throughput: 29.24 MB/sec

Concurrency: 997.06

Successful transactions: 1789818

Failed transactions: 0

Longest transaction: 0.87

Shortest transaction: 0.00

错误率是0

 

测试客户端B:

 

** SIEGE 2.69

** Preparing 1000 concurrent users for battle.

The server is now under siege...

Lifting the server siege... done.

Transactions: 1768585 hits

Availability: 100.00 %

Elapsed time: 179.31 secs

Data transferred: 5201.65 MB

Response time: 0.10 secs

Transaction rate: 9863.28 trans/sec

Throughput: 29.01 MB/sec

Concurrency: 998.30

Successful transactions: 1768588

Failed transactions: 0

Longest transaction: 3.10

Shortest transaction: 0.03

错误率是0