nginx 报Too many open files

nginx 异常报 Too many open files

• 上周时，nginx已经报 Too many open files 当时把
  配置文件调整最大连接65535了，reload 重新加载nginx后不报错了。

cat    /proc/14921/limits |grep  "Max open file"
*		soft	nofile		65535
*		hard	nofile		65535

cat /etc/sysctl.conf 
net.ipv4.tcp_tw_reuse = 1 
net.ipv4.tcp_tw_recycle = 1

• 以为没问题了， 结果今天又报错Too many open files

• 这就奇怪了，配置都已经修改了，为什么还会报Too many open files了
• 最后看了下进程的limit限制，发现nginx主进程软限制是1024，硬限制是4096。

[root@xxxx log]# ps -ef|grep nginx|grep -v shutting
root      4334     1  0  2023 ?        00:00:02 nginx: master process /usr/local/nginx/sbin/nginx
nobody   12350  4334  0 3月15 ?       00:02:18 nginx: worker process
nobody   14918  4334  0 11:29 ?        00:00:01 nginx: worker process
nobody   14919  4334  0 11:29 ?        00:00:00 nginx: worker process
nobody   14920  4334  0 11:29 ?        00:00:01 nginx: worker process
nobody   14921  4334  0 11:29 ?        00:00:01 nginx: worker process
nobody   14922  4334  0 11:29 ?        00:00:01 nginx: worker process
nobody   14923  4334  0 11:29 ?        00:00:02 nginx: worker process
nobody   14924  4334  0 11:29 ?        00:00:01 nginx: worker process
nobody   14925  4334  0 11:29 ?        00:00:03 nginx: worker process
root     21621 16905  0 11:46 pts/3    00:00:00 grep --color=auto nginx

[root@xxx log]#  cat /proc/4334/limits
Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            seconds   
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            8388608              unlimited            bytes     
Max core file size        0                    unlimited            bytes     
Max resident set          unlimited            unlimited            bytes     
Max processes             124951               124951               processes 
Max open files            1024                 4096                 files     

[ccodrunner@xxxx ]$ cat    /proc/14921/limits |grep  "Max open file"
Max open files            65535                65535                files   

• 看nginx主进程是2023年启动时间，可能是在启动nginx时，当时文件打开数为进行配置时，未重启nginx主进程，才导致最近业务量增加，经常出现无法创建打开文件数情况。
• 最后等晚上重启nginx主进程，待记录观察。

• 如果重启还是显示4096，则做如下操作

vim /etc/systemd/system.conf
DefaultLimitNOFILE=65535

systemctl daemon-reexec
然后再重启nginx服务

[root@xxxx sbin]# cat /proc/10311/limits |grep files
Max open files            65535                65535                files