VxLAN数据中心L2/L3互通（端到端）

VxLAN数据中心端到端方式实现L2/L3互连，这种实现方式可以使数据中心属于同一个EVPN-VXLAN域，相较于hand-off方式通过端到端实现数据中心L2互连可以满足Mac mobility、ARP suppression等特性。

实现思路

DC1的Border-Leaf和DC2的Border-Leaf之间运行EBGP EVPN协议传递MAC路由或者主机路由信息，不改变MAC路由或者主机路由的下一跳地址，从而在跨数据中心的VTEP之间建立端到端VXLAN隧道。

配置步骤

1.配置各节点IP地址。
2.配置路由实现各节点之间的互通。
3.在DC1和DC2内分别创建EBGP邻居，配置BGP EVPN协议创建VXLAN隧道。
4.WAN-PE之间L3互通，使两个DC之间的Lo0能互通。
5.DC1-BL与DC2-BL之间创建EBGP EVPN邻居。

拓扑图

这里起了两个vlan，并配置分布式网关，
VPC6、VPC35 两个主机在vlan100；veos55和veos56在vlan200，两个vlan属于同租户；
每个DC内有一台leaf、两台border-leaf做malg，剩下一台leaf忽略了。
WAN模拟相对简单只是L3互连，没有涉及到MPLS。

数据准备

IP

DC1设备	接口	IP	对端设备	接口	IP
DC1-Spine1	eth1	10.1.2.1/30	DC1-Leaf1	eth1	10.1.2.2/30
DC1-Spine1	eth2	10.1.3.1/30	DC1-Leaf2	eth1	10.1.3.2/30
DC1-Spine1	eth3	10.1.4.1/30	DC1-BL1	eth1	10.1.4.2/30
DC1-Spine1	eth4	10.1.5.1/30	DC1-BL2	eth1	10.1.5.2/30
DC1-Spine2	eth1	10.2.2.1/30	DC1-Leaf1	eth2	10.2.2.2/30
DC1-Spine2	eth2	10.2.3.1/30	DC1-Leaf2	eth2	10.2.3.2/30
DC1-Spine2	eth3	10.2.4.1/30	DC1-BL1	eth2	10.2.4.2/30
DC1-Spine2	eth4	10.2.5.1/30	DC1-BL2	eth2	10.2.5.2/30
DC1-BL1	eth4	172.16.0.1/30	DC1-BL2	eth4	172.16.0.2/30
DC1-Spine1	Lo0	10.0.0.1/32	DC1-Spine2	Lo0	10.0.0.2/32
DC1-Leaf1	Lo0	10.0.0.3/32	DC1-Leaf2	Lo0	10.0.0.4/32
DC1-BL1	Lo0	10.0.0.5/32	DC1-BL2	Lo0	10.0.0.6/32
DC1-BL1/2	Lo1	1.1.1.1/32 BL1&2Lo1相同
VPC6	eth0	192.168.100.6	VPC34	eth0	192.168.100.34
veos55	Po3	192.168.200.55

WAN设备	接口	IP	对端设备	接口	IP
PE1	Po2	172.17.0.0/31	PE2	Po2	172.17.0.1/31
PE1	eth1	172.31.0.1/31	DC1-BL1	eth3	172.31.0.0/31
PE1	eth2	172.32.0.1/31	DC1-BL2	eth3	172.32.0.0/31
PE2	eth1	172.33.0.0/31	DC2-BL1	eth3	172.33.0.1/31
PE2	eth2	172.34.0.0/31	DC2-BL2	eth3	172.34.0.1/31
PE2	Lo0	10.0.0.26/32	PE2	Lo0	10.0.0.50/32

DC2设备	接口	IP	对端设备	接口	IP
DC2-Spine1	eth1	20.1.1.1/30	DC2-BL1	eth1	20.1.1.2/30
DC2-Spine1	eth2	20.1.2.1/30	DC2-BL2	eth1	20.1.2.2/30
DC2-Spine1	eth3	20.1.3.1/30	DC2-Leaf1	eth1	20.1.3.2/30
DC2-Spine1	eth4	20.1.4.1/30	DC2-Leaf2	eth1	20.1.4.2/30
DC2-Spine2	eth1	20.2.1.1/30	DC2-BL1	eth2	20.2.1.2/30
DC2-Spine2	eth2	20.2.2.1/30	DC2-BL2	eth2	20.2.2.2/30
DC2-Spine2	eth3	20.2.3.1/30	DC2-Leaf1	eth2	20.2.3.2/30
DC2-Spine2	eth4	20.2.4.1/30	DC2-Leaf2	eth2	20.2.4.2/30
DC2-BL1	eth4	192.168.2.1/30	DC1-BL2	eth4	192.168.2.2/30
DC2-Spine1	Lo0	10.0.0.51/32	DC2-Spine2	Lo0	10.0.0.52/32
DC2-Leaf1	Lo0	10.0.0.53/32	DC2-Leaf2	Lo0	10.0.0.54/32
DC2-BL1	Lo0	10.0.0.28/32	DC1-BL2	Lo0	10.0.0.29/32
DC2-BL1/2	Lo1	2.2.2.2/32 BL1&2Lo1相同
VPC35	eth0	192.168.100.35	VPC7	eth0	192.168.100.7
veos56	Po3	192.168.200.55

AS

设备	AS	设备	AS
DC1-Spine1	100	DC1-Spine2	99
DC1-Leaf1	101	DC1-Leaf2	102
DC1-BL1	103	DC1-BL2	104
DCI-VTEP1	60026	DCI-VTEP2	65050
DC2-Spine1	65051	DC2-Spine2	65052
DC2-Leaf1	65053	DC2-Leaf2	65054
DC2-BL1	65028	DC2-BL2	65029

详细配置

每个角色的配置各举一个。

Spine，以DC1-Spine1为例：

service routing protocols model multi-agent
hostname DC1-Spine1
spanning-tree mode none
interface Ethernet1
mtu 9200
no switchport
ip address 10.1.2.1/30
interface Ethernet2
mtu 9200
switchport access vlan 1800
no switchport
ip address 10.1.3.1/30
interface Ethernet3
mtu 9200
no switchport
ip address 10.1.4.1/30
interface Ethernet4
mtu 9200
no switchport
ip address 10.1.5.1/30
interface Loopback0
ip address 10.0.0.1/32
ip routing
router bgp 100
router-id 10.0.0.1
no bgp default ipv4-unicast
maximum-paths 128
neighbor overlay peer group
neighbor overlay update-source Loopback0
neighbor overlay ebgp-multihop
neighbor overlay send-community extended
neighbor overlay maximum-routes 0
neighbor underlay peer group
neighbor underlay maximum-routes 0
neighbor 10.0.0.3 peer group overlay
neighbor 10.0.0.3 remote-as 101
neighbor 10.0.0.4 peer group overlay
neighbor 10.0.0.4 remote-as 102
neighbor 10.0.0.5 peer group overlay
neighbor 10.0.0.5 remote-as 103
neighbor 10.0.0.6 peer group overlay
neighbor 10.0.0.6 remote-as 104
neighbor 10.1.2.2 peer group underlay
neighbor 10.1.2.2 remote-as 101
neighbor 10.1.3.2 peer group underlay
neighbor 10.1.3.2 remote-as 102
neighbor 10.1.4.2 peer group underlay
neighbor 10.1.4.2 remote-as 103
neighbor 10.1.5.2 peer group underlay
neighbor 10.1.5.2 remote-as 104
address-family evpn
neighbor overlay activate
address-family ipv4
neighbor underlay activate
network 10.0.0.1/32

Leaf，以DC1-Leaf1为例：

service routing protocols model multi-agent
hostname DC1-Leaf1
spanning-tree mode none
vlan 100,200
vrf instance 100
interface Ethernet1
mtu 9200
no switchport
ip address 10.1.2.2/30
interface Ethernet2
no switchport
ip address 10.2.2.2/30
interface Ethernet3
mtu 9200
switchport access vlan 100
interface Loopback0
ip address 10.0.0.3/32
interface Vlan100
vrf 100
ip address virtual 192.168.100.254/24
interface Vlan200
vrf 100
ip address virtual 192.168.200.254/24
interface Vxlan1
vxlan source-interface Loopback0
vxlan udp-port 4789
vxlan vlan 100,200 vni 10100,10200
ip virtual-router mac-address 00:00:00:00:00:01
ip routing
ip routing vrf 100
router bgp 101
router-id 10.0.0.3
no bgp default ipv4-unicast
maximum-paths 128
neighbor overlay peer group
neighbor overlay update-source Loopback0
neighbor overlay ebgp-multihop
neighbor overlay send-community extended
neighbor overlay maximum-routes 0
neighbor underlay peer group
neighbor underlay maximum-routes 0
neighbor 10.0.0.1 peer group overlay
neighbor 10.0.0.1 remote-as 100
neighbor 10.0.0.2 peer group overlay
neighbor 10.0.0.2 remote-as 99
neighbor 10.1.2.1 peer group underlay
neighbor 10.1.2.1 remote-as 100
neighbor 10.2.2.1 peer group underlay
neighbor 10.2.2.1 remote-as 99
vlan-aware-bundle vlans-1
rd 1:101
route-target both 1:1
redistribute learned
vlan 100,200
address-family evpn
neighbor overlay activate
address-family ipv4
neighbor underlay activate
network 10.0.0.3/32
vrf 100
rd 100:1
route-target import evpn 100:100
route-target export evpn 100:100
redistribute connected

Border-Leaf，以DC1-BL1为例：

service routing protocols model multi-agent
hostname DC1-BL1
spanning-tree mode mstp
no spanning-tree vlan-id 4094
vlan 10,100,200
vlan 4094
name mlag
trunk group mlagpeer
vrf instance 100
interface Port-Channel3
mtu 9200
switchport trunk allowed vlan 100,200
switchport mode trunk
mlag 3
interface Ethernet1
mtu 9200
no switchport
ip address 10.1.4.2/30
interface Ethernet2
mtu 9200
no switchport
ip address 10.2.4.2/30
interface Ethernet3
mtu 9200
no switchport
ip address 172.31.0.0/31
interface Ethernet4
mtu 9200
switchport mode trunk
switchport trunk group mlagpeer
interface Loopback0
ip address 10.0.0.5/32
interface Loopback1
ip address 1.1.1.1/32
interface Vlan100
vrf 100
ip address virtual 192.168.100.254/24
interface Vlan200
vrf 100
ip address virtual 192.168.200.254/24
interface Vlan4094
ip address 172.16.0.1/30
interface Vxlan1
vxlan source-interface Loopback1
vxlan udp-port 4789
vxlan vlan 100,200 vni 10100,10200
ip virtual-router mac-address 00:00:00:00:00:01
ip routing
ip routing vrf 100
mlag configuration
domain-id mlag-domain
local-interface Vlan4094
peer-address 172.16.0.2
peer-link Ethernet4
dual-primary detection delay 5 action errdisable all-interfaces
router bgp 103
router-id 10.0.0.5
no bgp default ipv4-unicast
maximum-paths 128
neighbor overlay peer group
neighbor overlay update-source Loopback0
neighbor overlay ebgp-multihop
neighbor overlay send-community extended
neighbor overlay maximum-routes 0
neighbor underlay peer group
neighbor underlay maximum-routes 0
neighbor 10.0.0.1 peer group overlay
neighbor 10.0.0.1 remote-as 100
neighbor 10.0.0.2 peer group overlay
neighbor 10.0.0.2 remote-as 99
neighbor 10.0.0.28 peer group overlay
neighbor 10.0.0.28 remote-as 65028
neighbor 10.0.0.29 peer group overlay
neighbor 10.0.0.29 remote-as 65029
neighbor 10.1.4.1 peer group underlay
neighbor 10.1.4.1 remote-as 100
neighbor 10.2.4.1 peer group underlay
neighbor 10.2.4.1 remote-as 99
neighbor 172.16.0.2 remote-as 104
neighbor 172.31.0.1 peer group underlay
neighbor 172.31.0.1 remote-as 65026
vlan-aware-bundle vlans-1
rd 1:103
route-target both 1:1
redistribute learned
vlan 100,200
address-family evpn
neighbor overlay activate
address-family ipv4
neighbor underlay activate
neighbor 172.16.0.2 activate
network 1.1.1.1/32
network 10.0.0.5/32
vrf 100
rd 100:3
route-target import evpn 100:100
route-target export evpn 100:100
redistribute connected

WAN-PE，以PE1为例：

service routing protocols model multi-agent
hostname WAN-PE1
spanning-tree mode none
interface Port-Channel2
mtu 9200
no switchport
ip address 172.17.0.0/31
interface Ethernet1
mtu 9200
no switchport
ip address 172.31.0.1/31
interface Ethernet2
no switchport
ip address 172.32.0.1/31
interface Ethernet3
channel-group 2 mode active
interface Ethernet4
channel-group 2 mode active
interface Loopback0
ip address 10.0.0.26/32
ip routing
router bgp 65026
no bgp default ipv4-unicast
neighbor underlay-ebgp peer group
neighbor underlay-ebgp maximum-routes 0
neighbor 172.17.0.1 peer group underlay-ebgp
neighbor 172.17.0.1 remote-as 65050
neighbor 172.31.0.0 peer group underlay-ebgp
neighbor 172.31.0.0 remote-as 103
neighbor 172.32.0.0 peer group underlay-ebgp
neighbor 172.32.0.0 remote-as 104
address-family ipv4
neighbor underlay-ebgp activate
network 10.0.0.26/32

状态检查

DC1-BL1包括：DC内部BGP EVPN，两个DC-BL之间的BGP IPv4族，到WAN-PE1的BGP IPv4族，以及还有两个到DC2-BL的BGP EVPN 状态Established

DC1-Leaf1的远端vtep，2.2.2.2是DC2-BL，10.0.0.53是DC2-Leaf1，可以看到跨数据中心是端到端的隧道

DC1-Leaf1的mac table


DC1-Leaf1的arp table

DC1-Leaf1上看VPC35的MAC-IP路由，下一跳是DC2-Leaf1（10.0.0.53）

VPC6,ping 通本网段VPC35，ping通跨网段veos55、veos56

总结

DCI 端到端互连方案中，WAN-PE只是起到路由传递，和外层数据包转发的作用。
相比hand-off，端到端支持了Mac mobility、ARP suppression。