VXLAN 主机VTEP（EVPN）

EVE环境模拟搭建一个基于主机VTEP的VXLAN数据中心网络。
实验里vtep是在linux主机上，同时linux主机还得有路由功能使VTEP的端点IP可达，所以两台linux服务器需要安装FRR。
数据转发平面使用VXLAN封装；在控制平面我打算选择使用EVPN和OVN两种不同方式实现。
这次实验控制平面配置了EVPN，OVN放在下次实验里。

拓扑图

Spine、Leaf、服务器之间配置OSPF，使Underlay路由可达；

Overlay 采用EVPN，EVPN可以配置在多个位置：

1. 在服务器之间建立端到端EVPN，
2. 也可以建在服务器和Spine之间，
3. 或是 Spine至Leaf配置EVPN，Leaf至服务器配置EVPN。
   考虑到贴近真实场景，我在实验中选择第3种方式。这样在数据中心内增加服务器时，骨干部分不用操作，只需要服务器和TOR建立EVPN即可。

Linux9和Linux8作为宿主机，使用namespace模拟租户虚机。

安装frr

curl -O https://rpm.frrouting.org/repo/frr-7-repo-1-0.el7.noarch.rpm
yum install frr-7-repo-1-0.el7.noarch.rpm -y
yum install frr frr-pythontools -y

sed -i “s/=no/=yes/g” /etc/frr/daemons
cat /etc/frr/daemons，路由协议功能全部打开

systemctl enable frr && systemctl start frr && systemctl status frr
linux 主机上都安装好frr后就可以配置路由协议了。

在服务器上配置ns模拟租户虚机，配置VTEP

Linux9配置 ns

ip netns add ns1
ip link add veth11 type veth peer name veth12
ip link set veth12 netns ns1
ip link set veth11 up
ip netns exec ns1 ip link set veth12 address 00:00:00:00:00:02
ip netns exec ns1 ip link set veth12 up
ovs-vsctl add-port br-int veth11
ip netns exec ns1 ip addr add 192.168.2.20/24 dev veth12

vxlan vtep

brctl addbr br-tun123
ip link set br-tun123 up
ip link add vxlan123 type vxlan id 123 local 10.0.0.9 dstport 4789 nolearning
ip link set vxlan123 up
brctl addif br-tun123 vxlan123
brctl addif br-tun123 veth11

Linux8配置 ns

ip netns add ns1
ip link add veth11 type veth peer name veth12
ip link set veth12 netns ns1
ip link set veth11 up
ip netns exec ns1 ip link set veth12 address 00:00:00:00:00:01
ip netns exec ns1 ip link set veth12 up
ovs-vsctl add-port br-int veth11
ip netns exec ns1 ip addr add 192.168.2.10/24 dev veth12

vxlan vtep

brctl addbr br-tun123
ip link set br-tun123 up
ip link add vxlan123 type vxlan id 123 local 10.0.0.8 dstport 4789 nolearning
ip link set vxlan123 up
brctl addif br-tun123 vxlan123
brctl addif br-tun123 veth11

配置路由

Spine与Leaf之间Underlay配置OSPF，Overlay配置BGP EVPN

interface Ethernet2
no switchport
ip address 172.16.0.2/31
!
interface Ethernet3
no switchport
ip address 172.16.0.4/31
!
interface Ethernet5
no switchport
ip address 172.17.0.2/31
!
interface Ethernet6
no switchport
ip address 172.17.0.4/31
!
interface Loopback0
ip address 10.0.0.1/32
!
ip routing
!
router bgp 101
router-id 10.0.0.1
no bgp default ipv4-unicast
neighbor overlay peer group
neighbor overlay update-source Loopback0
neighbor overlay ebgp-multihop
neighbor overlay send-community extended
neighbor overlay maximum-routes 0
neighbor 10.0.0.3 peer group overlay
neighbor 10.0.0.3 remote-as 103
neighbor 10.0.0.4 peer group overlay
neighbor 10.0.0.4 remote-as 104
neighbor 10.0.0.66 peer group overlay
neighbor 10.0.0.66 remote-as 106
neighbor 10.0.0.77 peer group overlay
neighbor 10.0.0.77 remote-as 107
!
address-family evpn
neighbor overlay activate
!
router ospf 1
redistribute static
network 10.0.0.1/32 area 0.0.0.0
network 172.16.0.2/31 area 0.0.0.0
network 172.16.0.4/31 area 0.0.0.0
network 172.17.0.2/31 area 0.0.0.0
network 172.17.0.4/31 area 0.0.0.0
max-lsa 12000

Leaf3与Spine、Linux9之间Underlay配置OSPF，Overlay配置BGP EVPN，其余Leaf同理

interface Ethernet1
no switchport
ip address 172.16.0.3/31
!
interface Ethernet3
no switchport
ip address 172.16.9.1/30
!
interface Loopback0
ip address 10.0.0.3/32
!
ip routing
!
router bgp 103
router-id 10.0.0.1
no bgp default ipv4-unicast
neighbor overlay peer group
neighbor overlay update-source Loopback0
neighbor overlay ebgp-multihop
neighbor overlay send-community extended
neighbor overlay maximum-routes 0
neighbor 10.0.0.1 peer group overlay
neighbor 10.0.0.1 remote-as 101
neighbor 10.0.0.9 peer group overlay
neighbor 10.0.0.9 remote-as 109
!
address-family evpn
neighbor overlay activate
!
router ospf 1
network 10.0.0.3/32 area 0.0.0.0
network 172.16.0.2/31 area 0.0.0.0
network 172.16.9.0/30 area 0.0.0.0
max-lsa 12000

Linux9与Leaf3、Leaf6之间Underlay配置OSPF，Overlay配置BGP EVPN，Leaf8同理

router bgp 109
no bgp ebgp-requires-policy
no bgp default ipv4-unicast
neighbor 10.0.0.3 remote-as 103
neighbor 10.0.0.3 ebgp-multihop 3
neighbor 10.0.0.3 update-source 10.0.0.9
neighbor 10.0.0.66 remote-as 106
neighbor 10.0.0.66 ebgp-multihop 3
neighbor 10.0.0.66 update-source 10.0.0.9
!
address-family l2vpn evpn
neighbor 10.0.0.3 activate
neighbor 10.0.0.66 activate
advertise-all-vni
exit-address-family
!
router ospf
network 10.0.0.9/32 area 0
network 172.16.9.0/30 area 0
network 172.17.9.0/30 area 0

验证

租户互通ping

[root@hv9 ~]# ip netns exec ns1 bash
[root@hv9 ~]# ping 192.168.2.10
PING 192.168.2.10 (192.168.2.10) 56(84) bytes of data.
64 bytes from 192.168.2.10: icmp_seq=1 ttl=64 time=38.4 ms
64 bytes from 192.168.2.10: icmp_seq=2 ttl=64 time=30.6 ms

Underlay路由OSPF

spine1#show ip ospf neighbor
Neighbor ID Instance VRF Pri State Dead Time Address Interface
10.0.0.77 1 default 1 FULL/DR 00:00:37 172.17.0.5 Ethernet6
10.0.0.66 1 default 1 FULL/DR 00:00:35 172.17.0.3 Ethernet5
10.0.0.4 1 default 1 FULL/DR 00:00:29 172.16.0.5 Ethernet3
10.0.0.3 1 default 1 FULL/DR 00:00:30 172.16.0.3 Ethernet2

leaf3#sho ip ospf neighbor
Neighbor ID Instance VRF Pri State Dead Time Address Interface
10.0.0.1 1 default 1 FULL/BDR 00:00:33 172.16.0.2 Ethernet1
10.0.0.9 1 default 1 FULL/DR 00:00:30 172.16.9.2 Ethernet3

hv9# show ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface RXmtL RqstL DBsmL
10.0.0.3 1 Full/Backup 35.377s 172.16.9.1 eth0:172.16.9.2 0 0 0
10.0.0.66 1 Full/DR 33.954s 172.17.9.1 eth1:172.17.9.2 0 0 0

Overlay路由 EVPN

spine1#show bgp summary
BGP summary information for VRF default
Router identifier 10.0.0.1, local AS number 101
Neighbor AS Session State AFI/SAFI AFI/SAFI State NLRI Rcd NLRI Acc

---

10.0.0.3 103 Established L2VPN EVPN Negotiated 1 1
10.0.0.4 104 Established L2VPN EVPN Negotiated 1 1
10.0.0.66 106 Established L2VPN EVPN Negotiated 1 1
10.0.0.77 107 Established L2VPN EVPN Negotiated 1 1

leaf3#show bgp summary
BGP summary information for VRF default
Router identifier 10.0.0.1, local AS number 103
Neighbor AS Session State AFI/SAFI AFI/SAFI State NLRI Rcd NLRI Acc

---

10.0.0.1 101 Established L2VPN EVPN Negotiated 1 1
10.0.0.9 109 Established L2VPN EVPN Negotiated 1 1

hv9# show bgp summary

L2VPN EVPN Summary:
BGP router identifier 10.0.0.9, local AS number 109 vrf-id 0
BGP table version 0
RIB entries 3, using 576 bytes of memory
Peers 2, using 43 KiB of memory

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt
10.0.0.3 4 103 96 99 0 0 0 01:07:15 1 2
10.0.0.66 4 106 82 78 0 0 0 01:03:25 1 2

Total number of neighbors 2

Linux9 Underlay路由负载，其中一条上联中断VTEP地址仍可达

[root@hv9 ~]# ip route
10.0.0.1 proto 188 metric 20
nexthop via 172.16.9.1 dev eth0 weight 1
nexthop via 172.17.9.1 dev eth1 weight 1
10.0.0.2 proto 188 metric 20
nexthop via 172.16.9.1 dev eth0 weight 1
nexthop via 172.17.9.1 dev eth1 weight 1
10.0.0.3 via 172.16.9.1 dev eth0 proto 188 metric 20
10.0.0.4 proto 188 metric 20
nexthop via 172.16.9.1 dev eth0 weight 1
nexthop via 172.17.9.1 dev eth1 weight 1
10.0.0.5 proto 188 metric 20
nexthop via 172.16.9.1 dev eth0 weight 1
nexthop via 172.17.9.1 dev eth1 weight 1
10.0.0.6 proto 188 metric 20
nexthop via 172.16.9.1 dev eth0 weight 1
nexthop via 172.17.9.1 dev eth1 weight 1
10.0.0.7 proto 188 metric 20
nexthop via 172.16.9.1 dev eth0 weight 1
nexthop via 172.17.9.1 dev eth1 weight 1
10.0.0.8 proto 188 metric 20
nexthop via 172.16.9.1 dev eth0 weight 1
nexthop via 172.17.9.1 dev eth1 weight 1
10.0.0.55 proto 188 metric 20
nexthop via 172.16.9.1 dev eth0 weight 1
nexthop via 172.17.9.1 dev eth1 weight 1
10.0.0.66 via 172.17.9.1 dev eth1 proto 188 metric 20
10.0.0.77 proto 188 metric 20
nexthop via 172.16.9.1 dev eth0 weight 1
nexthop via 172.17.9.1 dev eth1 weight 1

MAC路由

hv9# show bgp l2vpn evpn
BGP table version is 35, local router ID is 10.0.0.9
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-1 prefix: [1]:[ESI]:[EthTag]:[IPlen]:[VTEP-IP]
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]

Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 10.0.0.8:2
*> [2]:[0]:[48]:[00:00:00:00:00:01]
10.0.0.8 0 103 101 104 108 i
RT:108:123 ET:8
*/ 10.0.0.8 0 106 101 104 108 i
RT:108:123 ET:8
*> [3]:[0]:[32]:[10.0.0.8]
10.0.0.8 0 103 101 107 108 i
RT:108:123 ET:8
*/ 10.0.0.8 0 106 101 107 108 i
RT:108:123 ET:8
Route Distinguisher: 10.0.0.9:2
*> [2]:[0]:[48]:[00:00:00:00:00:02]
10.0.0.9 32768 i
ET:8 RT:109:123
*> [3]:[0]:[32]:[10.0.0.9]
10.0.0.9 32768 i
ET:8 RT:109:123

Displayed 4 out of 6 total prefixes