昨天遇到一起服务器高延迟案例,这里总结一下,供大家参考!
然后有人反应网站打不开,直接内网ping服务器发现到服务器的延迟变化非常大,掉包严重,过一会又变回100多毫秒,然后又恢复正常,延迟变化非常大!
第一时间先查看其它同类网站访问和延迟是否正常,经查,均正常,说明整体带宽和外部环境正常,可能是服务器本身的问题,使用top命令查看cpu和内存使用情况均正常。联想到此服务器为近期新上的一台服务器,检查网卡流量信息
发现事发时段网卡流量在10M
检查此服务器网卡信息
#mii-tool -v eth0 eth0: negotiated 100baseTx-FD, link ok product info: vendor 00:50:ef, model 60 rev 8 basic mode: autonegotiation enabled basic status: autonegotiation complete, link ok capabilities: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD advertising: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control link partner: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD #ethtool eth0 Settings for eth0: Supported ports: [ TP ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Supports auto-negotiation: Yes Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Advertised auto-negotiation: Yes Speed: 100Mb/s Duplex: Full Port: Twisted Pair PHYAD: 1 Transceiver: internal Auto-negotiation: on Supports Wake-on: g Wake-on: d Link detected: yes
发现mii-tool和ethtool均显示网卡速率为100Mb/s,查资料后发现mii-tool支持的只到100M,已经废弃,ethtool为什么协商的速度为100M呢,服务器信息和交换机上硬件查看后均为千兆全双工,可能为连接介质的问题,更换千兆线缆后,再查看
#ethtool eth0 Settings for eth0: Supported ports: [ TP ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Supports auto-negotiation: Yes Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Advertised auto-negotiation: Yes Speed: 1000Mb/s Duplex: Full Port: Twisted Pair PHYAD: 1 Transceiver: internal Auto-negotiation: on Supports Wake-on: g Wake-on: d Link detected: yes
速率已经变为1000,网卡丢包情况消失了,可以确认为网卡速率为百兆,当流量达到上限后,无法再处理更多请求,则会出现丢包,ping不通现象!
也可以使用iptraf软件查看实时网络流量,直接使用yum即可安装
对访问流量太大的客户端可以通过iptables进行处理,也可以通过分析web访问日志来处理
转载于:https://blog.51cto.com/francis198/1785088