题目:R32的loopback0不能访问ISIS的部分网络。(R32的loopback0不能访问R31路由器所有端口,不能访问R34 G0/0/1口)
一、故障根因判断
根因是:AR28的g0/0/2接口outbound方向做了针对源地址是AR32的Loopback0接口的报文过滤行为。
二、故障分析
2.1 故障重现
使用ping命令测试,测试结果如下:
-----------------------------------------------------------------------------------------------------------------------------------------------------
<AR32>ping -a 10.5.1.32 10.5.14.34
PING 10.5.14.34: 56 data bytes, press CTRL_C to break
Request time out
Request time out
Request time out
Request time out
Request time out
--- 10.5.14.34 ping statistics ---
5 packet(s) transmitted
0 packet(s) received
100.00% packet loss
-----------------------------------------------------------------------------------------------------------------------------------------------------
<AR32>ping -a 10.5.1.32 10.5.1.31
PING 10.5.1.31: 56 data bytes, press CTRL_C to break
Request time out
Request time out
Request time out
Request time out
Request time out
--- 10.5.1.31 ping statistics ---
5 packet(s) transmitted
0 packet(s) received
100.00% packet loss
-----------------------------------------------------------------------------------------------------------------------------------------------------
<AR32>ping -a 10.5.1.32 10.5.14.31
PING 10.5.14.31: 56 data bytes, press CTRL_C to break
Request time out
Request time out
Request time out
Request time out
Request time out
--- 10.5.14.31 ping statistics ---
5 packet(s) transmitted
0 packet(s) received
100.00% packet loss
-----------------------------------------------------------------------------------------------------------------------------------------------------
<AR32>ping -a 10.5.1.32 10.5.130.31
PING 10.5.130.31: 56 data bytes, press CTRL_C to break
Request time out
Request time out
Request time out
Request time out
Request time out
--- 10.5.130.31 ping statistics ---
5 packet(s) transmitted
0 packet(s) received
100.00% packet loss
-----------------------------------------------------------------------------------------------------------------------------------------------------
对以上测试结果分析得出,AR32的Loopback0接口地址不能访问AR34的G0/0/1和AR31的所有接口地址,其他路由器的接口地址都可以正常通信,输出结果省略。所以初步怀疑是AR32上缺少去往这些接口地址的路由信息,需要进一步判断。
2.2 在AR32使用dis ip routing-table 命令查看其路由表,结果如下所示:
<AR32>dis ip routing-table
Route Flags: R - relay, D - download to fib
-----------------------------------------------------------------------------------------------------------------------------------------------------
Routing Tables: Public
Destinations : 18 Routes : 18
Destination/Mask Proto Pre Cost Flags NextHop Interface
10.5.1.27/32 OSPF 10 2 D 10.5.239.28 GigabitEthernet0/0/0
10.5.1.28/32 OSPF 10 1 D 10.5.239.28 GigabitEthernet0/0/0
10.5.1.30/32 O_ASE 150 1 D 10.5.239.28 GigabitEthernet0/0/0
10.5.1.31/32 O_ASE 150 1 D 10.5.239.28 GigabitEthernet0/0/0
10.5.1.32/32 Direct 0 0 D 127.0.0.1 LoopBack0
10.5.1.34/32 O_ASE 150 1 D 10.5.239.28 GigabitEthernet0/0/0
10.5.14.0/24 O_ASE 150 1 D 10.5.239.28 GigabitEthernet0/0/0
10.5.34.0/24 O_ASE 150 1 D 10.5.239.28 GigabitEthernet0/0/0
10.5.128.0/24 OSPF 10 2 D 10.5.239.28 GigabitEthernet0/0/0
10.5.129.0/24 O_ASE 150 1 D 10.5.239.28 GigabitEthernet0/0/0
10.5.130.0/24 O_ASE 150 1 D 10.5.239.28 GigabitEthernet0/0/0
10.5.239.0/24 Direct 0 0 D 10.5.239.32 GigabitEthernet0/0/0
10.5.239.32/32 Direct 0 0 D 127.0.0.1 GigabitEthernet0/0/0
10.5.239.255/32 Direct 0 0 D 127.0.0.1 GigabitEthernet0/0/0
127.0.0.0/8 Direct 0 0 D 127.0.0.1 InLoopBack0
127.0.0.1/32 Direct 0 0 D 127.0.0.1 InLoopBack0
127.255.255.255/32 Direct 0 0 D 127.0.0.1 InLoopBack0
255.255.255.255/32 Direct 0 0 D 127.0.0.1 InLoopBack0
-----------------------------------------------------------------------------------------------------------------------------------------------------
通过以上的输出结果表明,发现AR32中存在ISIS网络中所有路由条目。因此可以确定AR32的路由控制平面不存在问题,怀疑是AR34与AR31不存在AR32的LoopBack0接口地址的路由信息,需要进一步检查。
2.3 在AR34和AR31上使用命令进行检查,检查结果如下所示:
-----------------------------------------------------------------------------------------------------------------------------------------------------
<AR34>dis ip routing-table
Route Flags: R - relay, D - download to fib
-------------------------------------------------
Routing Tables: Public
Destinations : 16 Routes : 17
Destination/Mask Proto Pre Cost Flags NextHop Interface
0.0.0.0/0 ISIS-L1 15 10 D 10.5.14.31 GigabitEthernet0/0/1
ISIS-L1 15 10 D 10.5.34.30 GigabitEthernet0/0/0
10.5.1.30/32 ISIS-L1 15 10 D 10.5.34.30 GigabitEthernet0/0/0
10.5.1.31/32 ISIS-L1 15 10 D 10.5.14.31 GigabitEthernet0/0/1
10.5.1.34/32 Direct 0 0 D 127.0.0.1 LoopBack0
10.5.14.0/24 Direct 0 0 D 10.5.14.34 GigabitEthernet0/0/1
10.5.14.34/32 Direct 0 0 D 127.0.0.1 GigabitEthernet0/0/1
10.5.14.255/32 Direct 0 0 D 127.0.0.1 GigabitEthernet0/0/1
10.5.34.0/24 Direct 0 0 D 10.5.34.34 GigabitEthernet0/0/0
10.5.34.34/32 Direct 0 0 D 127.0.0.1 GigabitEthernet0/0/0
10.5.34.255/32 Direct 0 0 D 127.0.0.1 GigabitEthernet0/0/0
10.5.129.0/24 ISIS-L1 15 20 D 10.5.34.30 GigabitEthernet0/0/0
10.5.130.0/24 ISIS-L1 15 20 D 10.5.14.31 GigabitEthernet0/0/1
127.0.0.0/8 Direct 0 0 D 127.0.0.1 InLoopBack0
127.0.0.1/32 Direct 0 0 D 127.0.0.1 InLoopBack0
127.255.255.255/32 Direct 0 0 D 127.0.0.1 InLoopBack0
255.255.255.255/32 Direct 0 0 D 127.0.0.1 InLoopBack0
-----------------------------------------------------------------------------------------------------------------------------------------------------
以上结果表明,虽然R34路由表中不存在AR32的Loopback0的接口地址路由,但是存在两条缺省,因此可以排除AR34控制平面的问题。继续查询AR31的路由表。
<AR31>dis ip routing-table
Route Flags: R - relay, D - download to fib
-----------------------------------------------------
Routing Tables: Public
Destinations : 20 Routes : 20
Destination/Mask Proto Pre Cost Flags NextHop Interface
10.5.1.27/32 ISIS-L2 15 74 D 10.5.130.28 GigabitEthernet0/0/2
10.5.1.28/32 ISIS-L2 15 74 D 10.5.130.28 GigabitEthernet0/0/2
10.5.1.30/32 ISIS-L1 15 20 D 10.5.14.34 GigabitEthernet0/0/1
10.5.1.31/32 Direct 0 0 D 127.0.0.1 LoopBack0
10.5.1.32/32 ISIS-L2 15 74 D 10.5.130.28 GigabitEthernet0/0/2
10.5.1.34/32 ISIS-L1 15 10 D 10.5.14.34 GigabitEthernet0/0/1
10.5.14.0/24 Direct 0 0 D 10.5.14.31 GigabitEthernet0/0/1
10.5.14.31/32 Direct 0 0 D 127.0.0.1 GigabitEthernet0/0/1
10.5.14.255/32 Direct 0 0 D 127.0.0.1 GigabitEthernet0/0/1
10.5.34.0/24 ISIS-L1 15 20 D 10.5.14.34 GigabitEthernet0/0/1
10.5.128.0/24 ISIS-L2 15 74 D 10.5.130.28 GigabitEthernet0/0/2
10.5.129.0/24 ISIS-L1 15 30 D 10.5.14.34 GigabitEthernet0/0/1
10.5.130.0/24 Direct 0 0 D 10.5.130.31 GigabitEthernet0/0/2
10.5.130.31/32 Direct 0 0 D 127.0.0.1 GigabitEthernet0/0/2
10.5.130.255/32 Direct 0 0 D 127.0.0.1 GigabitEthernet0/0/2
10.5.239.0/24 ISIS-L2 15 74 D 10.5.130.28 GigabitEthernet0/0/2
127.0.0.0/8 Direct 0 0 D 127.0.0.1 InLoopBack0
127.0.0.1/32 Direct 0 0 D 127.0.0.1 InLoopBack0
127.255.255.255/32 Direct 0 0 D 127.0.0.1 InLoopBack0
255.255.255.255/32 Direct 0 0 D 127.0.0.1 InLoopBack0
-----------------------------------------------------------------------------------------------------------------------------------------------------
以上查询结果表明,AR31的路由表中存在AR32的Loopback0接口地址路由信息,因此可以排除AR31的控制平面的问题,同时表明AR28上单点双向引入正常并且路由信息正常。
综上分析,AR28、AR32、AR31、AR34的控制平面不存在问题。同时也可以排除物理层和数据链路层的故障,因此故障定位在数据平面。因为AR32是通过AR28接口访问AR31的三个接口和AR32的G0/0/1接口的,因此怀疑某台路由器使用了基于目标地址为AR31的三个接口和AR34的G0/0/1接口的流量过滤策略。需要进一步判断。
2.4 在AR32上使用不带源地址的tracert命令,验证到达AR31的三个接口和AR34的G0/0/1接口的路径,测试结果如下:
-----------------------------------------------------------------------------------------------------------------------------------------------------
<AR32>tracert 10.5.1.31
traceroute to 10.5.1.31(10.5.1.31), max hops: 30 ,packet length: 40,press CTRL
_C to break
1 10.5.239.28 30 ms 20 ms 20 ms
2 10.5.130.31 60 ms 30 ms 30 ms
<AR32>tracert 10.5.1.14.31
Error: Unknown host 10.5.1.14.31.
<AR32>tracert 10.5.14.31
traceroute to 10.5.14.31(10.5.14.31), max hops: 30 ,packet length: 40,press CT
RL_C to break
1 10.5.239.28 20 ms 20 ms 30 ms
2 10.5.130.31 20 ms 20 ms 20 ms
<AR32>tracert 10.5.130.31
traceroute to 10.5.130.31(10.5.130.31), max hops: 30 ,packet length: 40,press
CTRL_C to break
1 10.5.239.28 30 ms 30 ms 20 ms
2 10.5.130.31 20 ms 30 ms 30 ms
<AR32>tracert 10.5.14.34
traceroute to 10.5.14.34(10.5.14.34), max hops: 30 ,packet length: 40,press CT
RL_C to break
1 10.5.239.28 20 ms 30 ms 20 ms
2 10.5.130.31 30 ms 20 ms 30 ms
3 10.5.14.34 40 ms 50 ms 30 ms
-----------------------------------------------------------------------------------------------------------------------------------------------------
以上结果被表明,AR28、AR31、AR34没有使用基于目标地址为AR31的三个接口和AR34的G0/0/1接口的流量过滤策略,怀疑是某台路由器使用了基于目标地址为AR3LoopBack0接口的流量过滤策略。需要进一步判断。
2.5 继续在AR32使用带源地址为Loopback0接口地址的tracert命令,验证到达AR31的三个接口和AR34的G0/0/1接口的路径,测试结果如下:
-----------------------------------------------------------------------------------------------------------------------------------------------------
<AR32>tracert -a 10.5.1.32 10.5.1.31
traceroute to 10.5.1.31(10.5.1.31), max hops: 30 ,packet length: 40,press CTRL
_C to break
1 10.5.239.28 20 ms 10 ms 1 ms
2 * * *
<AR32>tracert -a 10.5.1.32 10.5.14.31
traceroute to 10.5.14.31(10.5.14.31), max hops: 30 ,packet length: 40,press CT
RL_C to break
1 10.5.239.28 20 ms 20 ms 20 ms
2 * * *
<AR32>tracert -a 10.5.1.32 10.5.130.31
traceroute to 10.5.130.31(10.5.130.31)
, max hops: 30 ,packet length: 40,press CTRL_C to break
1 10.5.239.28 20 ms 10 ms 10 ms
2 * * *
<AR32>tracert -a 10.5.1.32 10.5.14.34
traceroute to 10.5.14.34(10.5.14.34), max hops: 30 ,packet length: 40,press CT
RL_C to break
1 10.5.239.28 20 ms 20 ms 20 ms
2 * * *
-----------------------------------------------------------------------------------------------------------------------------------------------------
以上测试结果表明,AR32的tracert报文都是在第二跳出现无法正常回显的现象,初步判断AR28和AR31之间存在丢包现象。
2.6 在AR32上以LoopBack0接口地址作为源地址,10.5.130.31作为目标地址做ping测试,并且同时在AR31上查看g0/0/2接口的相关信息,结果如下所示:
-----------------------------------------------------------------------------------------------------------------------------------------------------
<AR32>ping -a 10.5.1.32 10.5.130.31
PING 10.5.130.31: 56 data bytes, press CTRL_C to break
Request time out
Request time out
Request time out
Request time out
Request time out
--- 10.5.130.31 ping statistics ---
5 packet(s) transmitted
0 packet(s) received
100.00% packet loss
-----------------------------------------------------------------------------------------------------------------------------------------------------
<AR31>dis interface GigabitEthernet 0/0/2
。。。。。。
Input: 685 packets, 956108 bytes
Unicast: 18, Multicast: 663
Broadcast: 4, Jumbo: 0
Discard: 0, Total Error: 0
。。。。。。
-----------------------------------------------------------------------------------------------------------------------------------------------------
经过多次重复的ping测试和观察AR31的G0/0/2接口的信息,发现接口状态是UP,但是接口的Input方向收到的Unicast单播报文数量却始终没有增加,所以可以判断AR28没有将数据包发送出去。但是不能确定AR31和AR34是否做了流量过滤,需要进一步排查。
2.7 在AR31和AR34上使用命令检查流量过滤必须使用的ACL,检查结果如下:
-----------------------------------------------------------------------------------------------------------------------------------------------------
<AR31>dis acl all
Total quantity of nonempty ACL number is 0
<AR32>dis acl all
Total quantity of nonempty ACL number is 0
-----------------------------------------------------------------------------------------------------------------------------------------------------
检查结果显示,AR31和AR32上无相关ACL,所以排除在AR31和AR34上做流量过滤的可能。
2.8 在AR34使用tracert路由跟踪AR32的Loopback0,测试针对反向数据包是否有过滤行为,测试结果如下:
-----------------------------------------------------------------------------------------------------------------------------------------------------
<AR34>tracert -a 10.5.14.34 10.5.1.32
traceroute to 10.5.1.32(10.5.1.32), max hops: 30 ,packet length: 40,press CTRL
_C to break
1 10.5.34.30 30 ms 20 ms 10 ms
2 10.5.129.28 20 ms 30 ms 10.5.130.28 30 ms
3 10.5.239.32 40 ms 40 ms 30 ms
-----------------------------------------------------------------------------------------------------------------------------------------------------
测试结果表明,路由跟踪是成功的,所以可以排除AR34-AR32反方向的数据报过滤。
2.9 综上分析:
2.1-2.3步骤可以判断出路由器不存在控制平面的问题;
2.4步骤可以判断不存在对报文目的地址的过滤行为;
2.5-2.8步骤可以判断AR28的g0/0/2接口outbound方向做了针对源地址是AR32的Loopback0接口的报文过滤行
三、故障处理
3.1 登录AR28执行以下命令:
sys
interface g0/0/2
undo traffic-filter outbound
undo traffic-policy outbound
执行完以上命令后,在AR32上执行以下命令进行测试:
ping -a 10.5.1.32 10.5.1.34
ping -a 10.5.1.32 10.5.1.31
ping -a 10.5.1.32 10.5.130.31
ping -a 10.5.1.32 10.5.14.31
3.2 其他高可能性: 如果在ping测试后,某些地址还是无法访问,则存在使用高级ACL过滤报文的可能,需要删除这些高级ACL。
3.3 如果执行完上述操作仍然不能恢复故障,则需要客户提供完整的设备配置或者前往现场进行排查,并拨打华为400服务热线请求华为TAC专家协助解决,谢谢!