5.6、不同网段,不同Leaf下服务器Ping(继续)
172.16.1.20 首先发起Ping 10.10.10.10。
这是不同网段之间的Ping,以下是分析:
1)172.16.1.20发起Ping 10.10.10.10,但没有实际发出Ping包,因为此时172.16.1.20没有网关172.16.1.1的mac地址,不具备Ping报文封装条件。172.16.1.20发arp1,得到172.16.1.1的mac1。
2)Leaf2的分布式网关172.16.1.1收到发给自己的arp包,会触发bgp evpn type2 irb route update(注意必须是发给自己的arp报文,即解析网关ip地址的arp报文,才会触发),通告rd/rt/mac1/ip1/L2vni/L3vni,即30:1/(100:10, 200:10)/mac1/172.16.1.20/8000/100。(看之前博客的EVPN相关配置数据)
3)Leaf1通过RR收到update,检查RT,和evpn instance/l3vpn instance的RT去匹配,其中:
100:10匹配了evpn 100的接收rt,同时L2VNI 8000和自己的vni匹配,这条bgp update的mac1会注入自己的bgp evpn表项(evpn-instance 100)中,最终mac1会注入mac表项(dis mac-address查看);
200:10匹配了l3vpn vpn1的接收rt,同时L3VNI100和自己配置的L3VNI匹配,这条bgp update的172.16.1.20会注入l3vpn instance vpn1对应的自己内部的一个evpn instance(EVPN-Instance __RD_1_200_1__),最终172.16.1.20会注入L3VPN instance vpn1的路由表中。
[Leaf1]dis bgp evpn all routing-table mac-route
Local AS number : 100
BGP Local router ID is 2.2.2.2
Status codes: * - valid, > - best, d - damped, x - best external, a - add path,
h - history, i - internal, s - suppressed, S - Stale
Origin : i - IGP, e - EGP, ? - incomplete
EVPN address family:
Number of Mac Routes: 3
Route Distinguisher: 20:1
Network(EthTagId/MacAddrLen/MacAddr/IpAddrLen/IpAddr) NextHop
*> 0:48:0001-0001-0001:0:0.0.0.0 0.0.0.0
Route Distinguisher: 30:1
Network(EthTagId/MacAddrLen/MacAddr/IpAddrLen/IpAddr) NextHop
*>i 0:48:0001-0001-0001:0:0.0.0.0 30.30.30.30
*>i 0:48:5489-98e3-2ad7:32:172.16.1.20 30.30.30.30
EVPN-Instance 100:
Number of Mac Routes: 3
Network(EthTagId/MacAddrLen/MacAddr/IpAddrLen/IpAddr) NextHop
*> 0:48:0001-0001-0001:0:0.0.0.0 0.0.0.0
* i 30.30.30.30
*>i 0:48:5489-98e3-2ad7:32:172.16.1.20 30.30.30.30
EVPN-Instance __RD_1_200_1__:
Number of Mac Routes: 1
Network(EthTagId/MacAddrLen/MacAddr/IpAddrLen/IpAddr) NextHop
*>i 0:48:5489-98e3-2ad7:32:172.16.1.20 30.30.30.30
[Leaf1]
[Leaf1]dis mac-address
Flags: * - Backup
BD : bridge-domain Age : dynamic MAC learned time in seconds
-------------------------------------------------------------------------------
MAC Address VLAN/VSI/BD Learned-From Type Age
-------------------------------------------------------------------------------
5489-98e3-2ad7 -/-/100 30.30.30.30 dynamic -
5489-9896-70c5 -/-/200 GE1/0/1.2 dynamic -
5489-98e3-2ad7 -/-/100 30.30.30.30 dynamic -
5489-9896-70c5 -/-/200 GE1/0/1.2 dynamic -
-------------------------------------------------------------------------------
Total items: 4
[Leaf1]
[Leaf1]dis ip routing-table vpn-instance vpn1
Proto: Protocol Pre: Preference
Route Flags: R - relay, D - download to fib, T - to vpn-instance, B - black hole route
------------------------------------------------------------------------------
Routing Table : vpn1
Destinations : 10 Routes : 10
Destination/Mask Proto Pre Cost Flags NextHop Interface
10.10.10.0/24 Direct 0 0 D 10.10.10.1 Vbdif200
10.10.10.1/32 Direct 0 0 D 127.0.0.1 Vbdif200
10.10.10.255/32 Direct 0 0 D 127.0.0.1 Vbdif200
172.16.1.0/24 Direct 0 0 D 172.16.1.1 Vbdif100
172.16.1.1/32 Direct 0 0 D 127.0.0.1 Vbdif100
172.16.1.20/32 IBGP 255 0 RD 30.30.30.30 VXLAN
172.16.1.255/32 Direct 0 0 D 127.0.0.1 Vbdif100
182.1.1.0/24 IBGP 255 0 RD 40.40.40.40 VXLAN
182.1.1.1/32 IBGP 255 0 RD 40.40.40.40 VXLAN
255.255.255.255/32 Direct 0 0 D 127.0.0.1 InLoopBack0
[Leaf1]
4)获得网关mac1后,172.16.1.20发起Ping 10.10.10.10,ping包到达Leaf2,查找vbdif对应的路由表(网关在Leaf2配置为vbdif,该端口下有ip binding vpn-instance vpn1),10.10.10.0/24路由已经通过bgp evpn type5 update宣告过来了,所以Leaf2用L3VNI 100,把ping包封装为vxlan包,内层报文的源、目mac为两端VTEP的mac地址,目的mac是通过type5 update里面的router mac获得的。封装后的vxlan包,发往Leaf1。
5)Leaf1收到后,根据L3VNI 100查找对应的L3VPN instance vpn1的路由表,查询结果为需要从网关10.10.10.1发送出去,但此时网关不知道10.10.10.10的mac,于是发出arp2 request,并收到reply。此时ping包会timeout。
6) 发出第二个Ping包,到达Leaf1后,顺利发往10.10.10.10,10.10.10.10发icmp reply,到达Leaf1后,查看路由表,有172.16.1.20的主机路由(之前学到的),正常进行L3VNI vxlan封装,并发到Leaf2,到Leaf2后,发到172.16.1.20。
至此,完成一个正常Ping。
另外一个问题说明:
由于缺省情况下,arp timeout 20分钟,mac表 timeout 4分钟,
172.16.1.20ping 10.10.10.10,ping通后,过了4分钟以后,Leaf1的10.10.10.10的arp信息会存在,但10.10.10.10的mac信息已经删除。
[Leaf1]dis arp
ARP Entry Types: D - Dynamic, S - Static, I - Interface, O - OpenFlow, RD - Redirect
EXP: Expire-time VLAN:VLAN or Bridge Domain
IP ADDRESS MAC ADDRESS EXP(M) TYPE/VLAN INTERFACE VPN-INSTANCE
----------------------------------------------------------------------------------------
192.168.12.2 387d-c803-0100 I GE1/0/0
192.168.12.1 387d-c804-0101 5 D GE1/0/0
172.16.1.1 0001-0001-0001 I Vbdif100 vpn1
10.10.10.1 707b-e8da-5876 I Vbdif200 vpn1
10.10.10.10 5489-9896-70c5 9 D/BD200 GE1/0/1.2 vpn1
----------------------------------------------------------------------------------------
Total:5 Dynamic:2 Static:0 Interface:3 OpenFlow:0
Redirect:0
[Leaf1]
[Leaf1]dis mac
Flags: * - Backup
BD : bridge-domain Age : dynamic MAC learned time in seconds
-------------------------------------------------------------------------------
MAC Address VLAN/VSI/BD Learned-From Type Age
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Total items: 0
[Leaf1]
此时,再次从172.16.1.20 ping 10.10.10.10,ping包到达Leaf1后,根据arp表封装,此时对应Leaf1的二层转发来说,10.10.10.10的mac地址属于BUM中的unknow mac,会进行flooding操作。
通过Leaf1连接业务侧的端口抓包可以看出,ping包正常转发出去了(通过unknow mac flooding转发机制发出去的)。此时,由于10.10.10.10服务器中,网关的arp信息也timeout了,所以10.10.10.10收到ping包后,发出了arp包。