问题:
昨天监控系统报告一台Cisco 3550交换机工作不正常,在局域网内ping值竟然达到了300多ms。登上这台交换机查看日志如下,问题为内存不足。这台交换机一直工作很正常,出现内存不足极有可能是病毒造成的。登上监控系统查看了一下这台交换机各个端口的使用率,fa0/17端口的占用率竟然高达90%。去机房查看了此端口,其一直显示为橙色状态,并且闪烁频率很高。现在大致可以判断问题是由这个端口造成的,随即关闭这个端口,问题解决。
show logging
-Process= "Pool Manager", ipl= 0, pid= 5
-Traceback= 1C31BC 1C4810 17A2A0 1D56B4 1D5824 1E5D24 1E9350
000082: *Mar 1 00:38:51: %SYS-CLUSTER_MEMBER_3-2-MALLOCFAIL: Memory allocation of 1680 bytes failed from 0x17A29C, alignment 0
Pool: I/O Free: 1672 Cause: Not enough free memory
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Traceback= 1C31BC 1C4810 17A2A0 1D56B4 1D5824 1E5D24 1E9350
000082: *Mar 1 00:38:51: %SYS-CLUSTER_MEMBER_3-2-MALLOCFAIL: Memory allocation of 1680 bytes failed from 0x17A29C, alignment 0
Pool: I/O Free: 1672 Cause: Not enough free memory
Alternate Pool: None Free: 0 Cause: No Alternate pool
原因:
查看配线架-交换机端口连接表,找到相应的用户端口。问题找到了:这个用户端口连接了一台D-Link的小交换机,不知是什么人竟然把同一根网线的两头都插在了这个小交换机里造成了环路。环路所导致的广播风暴消耗了交换机大量的内存资源,最后导致其几乎瘫痪。
解决办法:
可以在用户端口上设置广播所占带宽的上限值。如下设置fa0/17端口的广播所占带宽最大为20%。这样万一发生广播风暴,这个设置可以降低其影响。
storm-control broadcast level 20
查看各个端口的设置如下:
show storm-control
Interface Filter State Upper Lower Current
--------- ------------- ----------- ----------- -----------
Fa0/1 inactive 100.00% 100.00% N/A
Fa0/2 inactive 100.00% 100.00% N/A
Fa0/3 inactive 100.00% 100.00% N/A
Fa0/4 inactive 100.00% 100.00% N/A
Fa0/5 inactive 100.00% 100.00% N/A
Fa0/6 inactive 100.00% 100.00% N/A
Fa0/7 inactive 100.00% 100.00% N/A
Fa0/8 inactive 100.00% 100.00% N/A
Fa0/9 inactive 100.00% 100.00% N/A
Fa0/10 inactive 100.00% 100.00% N/A
Fa0/11 inactive 100.00% 100.00% N/A
Fa0/12 inactive 100.00% 100.00% N/A
Fa0/13 inactive 100.00% 100.00% N/A
Fa0/14 inactive 100.00% 100.00% N/A
Fa0/15 inactive 100.00% 100.00% N/A
Fa0/16 inactive 100.00% 100.00% N/A
Fa0/17 Forwarding 20.00% 20.00% 0.00%
Fa0/18 inactive 100.00% 100.00% N/A
Fa0/19 inactive 100.00% 100.00% N/A
Fa0/20 inactive 100.00% 100.00% N/A
Fa0/21 inactive 100.00% 100.00% N/A
Fa0/22 inactive 100.00% 100.00% N/A
Fa0/23 inactive 100.00% 100.00% N/A
Fa0/24 inactive 100.00% 100.00% N/A
Gi0/1 inactive 100.00% 100.00% N/A
Gi0/2 inactive 100.00% 100.00% N/A
--------- ------------- ----------- ----------- -----------
Fa0/1 inactive 100.00% 100.00% N/A
Fa0/2 inactive 100.00% 100.00% N/A
Fa0/3 inactive 100.00% 100.00% N/A
Fa0/4 inactive 100.00% 100.00% N/A
Fa0/5 inactive 100.00% 100.00% N/A
Fa0/6 inactive 100.00% 100.00% N/A
Fa0/7 inactive 100.00% 100.00% N/A
Fa0/8 inactive 100.00% 100.00% N/A
Fa0/9 inactive 100.00% 100.00% N/A
Fa0/10 inactive 100.00% 100.00% N/A
Fa0/11 inactive 100.00% 100.00% N/A
Fa0/12 inactive 100.00% 100.00% N/A
Fa0/13 inactive 100.00% 100.00% N/A
Fa0/14 inactive 100.00% 100.00% N/A
Fa0/15 inactive 100.00% 100.00% N/A
Fa0/16 inactive 100.00% 100.00% N/A
Fa0/17 Forwarding 20.00% 20.00% 0.00%
Fa0/18 inactive 100.00% 100.00% N/A
Fa0/19 inactive 100.00% 100.00% N/A
Fa0/20 inactive 100.00% 100.00% N/A
Fa0/21 inactive 100.00% 100.00% N/A
Fa0/22 inactive 100.00% 100.00% N/A
Fa0/23 inactive 100.00% 100.00% N/A
Fa0/24 inactive 100.00% 100.00% N/A
Gi0/1 inactive 100.00% 100.00% N/A
Gi0/2 inactive 100.00% 100.00% N/A
总结:
完善的监控系统,文档是快速分析,解决问题的利器。
本文出自 “面朝大海,春暖花开” 博客,谢绝转载!