SDN in Action: Practice VXLAN L2 Gateway with GNS3 and CSR1000v

  

薛国锋     xueguofeng2011@gmail.com

 

Virtual Extensible LAN (VXLAN) is a popular encapsulation protocol for running an overlay network, allowing us to create a Layer 2 network on top of a Layer 3 network. In a cloud computing environement, each tenant requires its own logical network and its network ID. Traditionally VLANs are used to  isolate apps and tenants, but its limitation of 4K network IDs severely restricts its adoption in a large cloud computing environment. The primary goal of VXLAN is to extend the network address space to 16 million by adding a 24-bit segment ID. In addtion, with the MAC-in-UDP encapsulation, millions of isolated Layer 2 VXLAN networks can co-exist on a common and long-distance Layer 3 infrastructure. Nowadays, VXLAN has been widely deployed not only in the DC and Cloud environment, but also in the wide area networks. 


Today we are going to build a mini-lab environment with GNS3 and CSR1000v, and get some hands-on experience with VXLAN.  Below is the network topology and desgin, we will create 2 Layer 2 VXLAN networks on top of a Layer 3 underlay network: 

-VXLAN 4096 with VPCS-1, VPCS-2 and VPCS-3, 10.110.0.0/16; 

-VXLAN 4097 with VPCS-4 and VPCS-5, 10.111.0.0/16.


test1.jpg.png


-          Setup the mini-lab environment

-          Understand VXLAN and its configuration on CSR1000v

-          Configure VXLAN L2 Gateway: Multicast Mode

-          Configure VXLAN L2 Gateway: Unicast Mode


Setup the mini-lab environment

 

Enable ‘Intel VT-x/EPT’ in the VM settings, and install Ubuntu:

test1.jpg.png

gset@ubuntu:~$ egrep -o '(vmx|svm)' /proc/cpuinfo

vmx

vmx

gset@ubuntu:~$ sudo apt-get update

gset@ubuntu:~$ sudo apt-get upgrade

gset@ubuntu:~$ sudo apt-get dist-upgrade

 

Install, reboot and run GNS3:

https://docs.gns3.com/1QXVIihk7dsOL7Xr7Bmz4zRzTsJ02wklfImGuHwTlaA4/index.html

gset@ubuntu:~$ sudo add-apt-repository ppa:gns3/ppa

gset@ubuntu:~$ sudo apt-get update

gset@ubuntu:~$ sudo apt-get install gns3-gui

// Should non-superusers be able to run GNS3 ? Yes

// Should non-superusers be able to capture packages ? Yes

gset@ubuntu:~$ reboot

gset@ubuntu:~$ gns3

 

Download the appliance file for CSR1000v and its image:

-           ‘cisco-csr1000v.gns3a’ ;

-           ‘csr1000v-universalk9.16.07.01-serial.qcow2’.

https://docs.gns3.com/appliances/cisco-csr1000v.html#appliance_supported

 

Import  ‘cisco-csr1000v.gns3a’ to GNS3 and run the appliance on the local computer:

test1.jpg.png


Create a new version for the appliance and import its image – ‘csr1000v-universalk9.16.07.01-serial.qcow2’:

test1.jpg.png


Customize the template of Qemu VM:

Edit > Preferences > QEMU > Qemu VMs > Edit 

test1.jpg.png


Install Konsole:

gset@ubuntu:~$ sudo apt-get install konsole

 

Config GNS3 to use Konsole insteadl gnome-terminal:

Edit > > Preferences > General > Console applications

gnome-terminal -t "%d" -e "telnet %h %p"  -----> /usr/bin/konsole --new-tab -p tabtitle=%d -e telnet %h %p

test1.jpg.png


Run CSR1000v the first time:  'CSR1000v - packages.conf'

Would you like to enter the initial configuration dialog? [yes/no]:no

Would you like to terminate autoinstall? [yes]:yes

Router>en

Router#conf

Configuring from terminal, memory, or network [terminal]?

Router(config)#hostname r1

r1(config)#exit

r1#write

Building configuration...

[OK]

r1#

r1#reload

Proceed with reload? [confirm]


Understand VXLAN and its configuration on CSR1000v


test1.jpg.png

test1.jpg.png


Based on the data plane MAC address learning and flooding of multidestination traffic using IP multicast or unicast, CSR1000v can act as a Layer 2 VXLAN gateway and provide support to bridge traffic across VxLAN segments and Ethernet/VLANs.

 

CSR1000v implements the Virtual Tunnel Endpoint (VTEP) function, which encapsulates the Layer 2 packet on the IP/UDP tunnel with the VXLAN header (VNI) information before sending it with multicast or unicast, and after receiving the packet, removes the VXLAN header, identifies the bridge domain under which the VNI is configured and then bridges the inner L2 packet to the Ethernet/VLAN side.

 

An NVE interface is the overlay interface that terminates VXLAN tunnels. Multiple VXLAN VNIs can be assoicated with a NVE interface.

Bridge domain represents a Layer 2 broadcast domain with a set of logical ports, and by default, it maintains its own forwarding database of MAC.

Service-instance is to bind some traffic based to their tag(s) or VLAN(s) to a service (L2 or L3), and connects a bridge domain to a physical or logical port.

 

The most commonly used VXLAN control plane is multicast. When configuring VXLAN with multicast control plane, every node configured with a VTEP for a VXLAN with certain VNI will join the same multicast group, and multicast configuration must support Any-Source Multicast (ASM). With the head-end replication, VXLAN can also support the unicast-only mode.

test1.jpg.png


Configure VXLAN L2 Gateway: Multicast Mode



Configuration for VPCS                

VXLAN: 4096

 

VPCS> set pcname VPCS-1

VPCS-1> ip 10.110.0.1 255.255.0.0

 

VPCS-2> set pcname VPCS-2

VPCS-2> ip 10.110.0.2 255.255.0.0

 

VPCS-3> set pcname VPCS-3

VPCS-3> ip 10.110.0.3 255.255.0.0

VXLAN: 4097

 

VPCS-4> set pcname VPCS-4

VPCS-4> ip 10.111.0.1 255.255.0.0

 

VPCS-5> set pcname VPCS-4

VPCS-5> ip 10.111.0.2 255.255.0.0

show ip

save                        

ping



Configuration for routers – r1,r2,r3,r4                                                                                    

r2#show run

Building configuration...

!

hostname r2

ip multicast-routing   distributed

!

interface Loopback100

 ip address 100.100.100.100 255.255.255.255

 ip pim sparse-mode

 ip ospf 100 area 100

!

interface GigabitEthernet2

 ip address 12.1.1.2 255.255.255.0

 ip pim sparse-mode

 ip ospf 100 area 100

 negotiation auto

 no mop enabled

 no mop sysid

!

interface GigabitEthernet3

 ip address 11.1.1.2 255.255.255.0

 ip pim sparse-mode

 ip ospf 100 area 100

 negotiation auto

 no mop enabled

 no mop sysid

!

interface GigabitEthernet4

 ip address 10.1.1.2 255.255.255.0

 ip pim sparse-mode

 ip ospf 100 area 100

 negotiation auto

 no mop enabled

 no mop sysid

!

router ospf 100

 router-id 2.2.2.2

!

ip pim bidir-enable

ip pim rp-address   100.100.100.100 bidir

r1#show run

Building configuration...

!

hostname r1

ip multicast-routing   distributed

!

bridge-domain 10

 member vni 4096

// Apply the service   instance

 member GigabitEthernet1 service-instance 10

!

bridge-domain 11

 member vni 4097

// Apply the service   instance

member GigabitEthernet2   service-instance 11

!

interface Loopback100

 ip address 10.10.10.10 255.255.255.255

 ip pim sparse-mode

 ip ospf 100 area 100

!

interface GigabitEthernet1

 no ip address

 negotiation auto

 no mop enabled

 no mop sysid

// define a service   instance

 service   instance 10 ethernet

    encapsulation untagged

!

interface GigabitEthernet2

 no ip address

 negotiation auto

 no mop enabled

 no mop sysid

// define a service   instance

 service   instance 11 ethernet

    encapsulation untagged

!

interface GigabitEthernet4

 ip address 10.1.1.1 255.255.255.0

 ip pim sparse-mode

 ip ospf 100 area 100

 negotiation auto

 no mop enabled

 no mop sysid

!

interface nve1

 no ip address

 source-interface Loopback100

// 1 multicast distribution   tree for multiple VNIs

 member vni   4096 mcast-group 225.1.1.1

 member vni   4097 mcast-group 225.1.1.1

 no mop enabled

 no mop sysid

!

router ospf 100

 router-id 1.1.1.1

!

ip pim bidir-enable

ip pim rp-address   100.100.100.100 bidir

r3#show run

Building configuration...

!

hostname r3

ip multicast-routing   distributed

!

bridge-domain 10

 member vni 4096

// Apply the service   instance

 member GigabitEthernet1 service-instance 10

!

bridge-domain 11

 member vni 4097

// Apply the service   instance

member GigabitEthernet2   service-instance 11

!

interface Loopback100

 ip address 11.11.11.11 255.255.255.255

 ip pim sparse-mode

 ip ospf 100 area 100

!

interface GigabitEthernet1

 no ip address

 negotiation auto

 no mop enabled

 no mop sysid

// define a service   instance

 service   instance 10 ethernet

    encapsulation untagged

!

interface GigabitEthernet2

 no ip address

 negotiation auto

 no mop enabled

 no mop sysid

// define a service   instance

 service   instance 11 ethernet

    encapsulation untagged

!

interface GigabitEthernet3

 ip address 11.1.1.1 255.255.255.0

 ip pim sparse-mode

 ip ospf 100 area 100

 negotiation auto

 no mop enabled

 no mop sysid

!

interface nve1

 no ip address

 source-interface Loopback100

// 1 multicast distribution   tree for multiple VNIs

 member vni   4096 mcast-group 225.1.1.1

 member vni   4097 mcast-group 225.1.1.1

 no mop enabled

 no mop sysid

!

router ospf 100

 router-id 3.3.3.3

!

ip pim bidir-enable

ip pim rp-address   100.100.100.100 bidir

r4#show run

Building configuration...

!

hostname r4

ip multicast-routing   distributed

!

bridge-domain 10

 member vni 4096

// Apply the service   instance

 member GigabitEthernet1 service-instance 10

!

 

 

 

 

 

 

interface Loopback100

 ip address 12.12.12.12 255.255.255.255

 ip pim sparse-mode

 ip ospf 100 area 100

!        

interface GigabitEthernet1

 no ip address

 negotiation auto

 no mop enabled

 no mop sysid

// define a service   instance

 service   instance 10 ethernet

    encapsulation untagged

!

 

 

 

 

 

 

 

 

 

interface GigabitEthernet2

 ip address 12.1.1.1 255.255.255.0

 ip pim sparse-mode

 ip ospf 100 area 100

 negotiation auto

 no mop enabled

 no mop sysid

!

interface nve1

 no ip address

 source-interface Loopback100

// 1 multicast distribution   tree for multiple VNIs

 member vni   4096 mcast-group 225.1.1.1

 

 

 no mop enabled

 no mop sysid

!

router ospf 100

 router-id 4.4.4.4

!

ip pim bidir-enable

ip pim rp-address   100.100.100.100 bidir

 

 

r1#show nve vni

r1#show nve peers

r1#show nve inter nve 1 detail

test1.jpg.png


r1#show ip route

r2#show ip route

test1.jpg.png

r1#show ip mroute

r2#show ip igmp interface loopback 100

r2#show ip pim neighbor

r2#show ip pim rp

test1.jpg.png

VPCS-1> ping 10.110.0.1

VPCS-1> ping 10.110.0.2

VPCS-1> ping 10.110.0.3

VPCS-4> ping 10.111.0.1

VPCS-4> ping 10.111.0.2

test1.jpg.png


Configure VXLAN L2 Gateway: Unicast Mode

Configuration for routers – r1,r2,r3,r4

r2#show run

Building configuration...

!

hostname r2

!

interface Loopback100

 ip address 100.100.100.100 255.255.255.255

 ip ospf 100 area 100

!

interface GigabitEthernet2

 ip address 12.1.1.2 255.255.255.0

 ip ospf 100 area 100

 negotiation auto

 no mop enabled

 no mop sysid

!

interface GigabitEthernet3

 ip address 11.1.1.2 255.255.255.0

 ip ospf 100 area 100

 negotiation auto

 no mop enabled

 no mop sysid

!

interface GigabitEthernet4

 ip address 10.1.1.2 255.255.255.0

 ip ospf 100 area 100

 negotiation auto

 no mop enabled

 no mop sysid

!

router ospf 100

 router-id 2.2.2.2

!

r1#show run

Building configuration...

!

hostname r1

!

bridge-domain 10

 member vni 4096

 member GigabitEthernet1 service-instance 10

!

bridge-domain 11

 member vni 4097

 member GigabitEthernet2 service-instance 11

!

interface Loopback100

 ip address 10.10.10.10 255.255.255.255

 ip ospf 100 area 100

!

interface GigabitEthernet1

 no ip address

 negotiation auto

 no mop enabled

 no mop sysid

 service instance 10 ethernet

  encapsulation untagged

!

interface GigabitEthernet2

 no ip address

 negotiation auto

 no mop enabled

 no mop sysid

 service instance 11 ethernet

  encapsulation untagged

!

interface GigabitEthernet4

 ip address 10.1.1.1 255.255.255.0

 ip ospf 100 area 100

 negotiation auto

 no mop enabled

 no mop sysid

!

interface nve1

 no ip address

 source-interface Loopback100

// Head-end replication   enables unicast-only mode

 member vni 4096

  ingress-replication 11.11.11.11

  ingress-replication 12.12.12.12

 member vni 4097

  ingress-replication 11.11.11.11

no mop enabled

 no mop sysid

!

router ospf 100

 router-id 1.1.1.1

!

r3#show run

Building configuration...

!

hostname r3

!

bridge-domain 10

 member vni 4096

 member GigabitEthernet1 service-instance 10

!

bridge-domain 11

 member vni 4097

 member GigabitEthernet2 service-instance 11

!

interface Loopback100

 ip address 11.11.11.11 255.255.255.255

 ip ospf 100 area 100

!

interface GigabitEthernet1

 no ip address

 negotiation auto

 no mop enabled

 no mop sysid

 service instance 10 ethernet

  encapsulation untagged

!

interface GigabitEthernet2

 no ip address

 negotiation auto

 no mop enabled

 no mop sysid

 service instance 11 ethernet

  encapsulation untagged

!

interface GigabitEthernet3

 ip address 11.1.1.1 255.255.255.0

 ip ospf 100 area 100

 negotiation auto

 no mop enabled

 no mop sysid

!

interface nve1

 no ip address

 source-interface Loopback100

// Head-end replication   enables unicast-only mode

 member vni 4096

  ingress-replication 10.10.10.10

  ingress-replication 12.12.12.12

 member vni 4097

  ingress-replication 10.10.10.10

 no mop enabled

 no mop sysid

!

router ospf 100

 router-id 3.3.3.3

!

 

r4#show run

Building configuration...

!

hostname r4

!

bridge-domain 10

 member vni 4096

 member GigabitEthernet1 service-instance 10

!

!

!

!

!

!

interface Loopback100

 ip address 12.12.12.12 255.255.255.255

 ip ospf 100 area 100

!        

interface GigabitEthernet1

 no ip address

 negotiation auto

 no mop enabled

 no mop sysid

 service instance 10 ethernet

  encapsulation untagged

!

 

 

 

 

 

 

 

 

interface GigabitEthernet2

 ip address 12.1.1.1 255.255.255.0

 ip ospf 100 area 100

 negotiation auto

 no mop enabled

 no mop sysid

!

interface nve1

 no ip address

 source-interface Loopback100

// Head-end replication   enables unicast-only mode

 member vni 4096

  ingress-replication 10.10.10.10

  ingress-replication 11.11.11.11

 

 

 no mop enabled

 no mop sysid

!

router ospf 100

 router-id 4.4.4.4

!

 

r1#show nve vni

r1#show nve peers

r1#show nve inter nve 1 detail

test1.jpg.png

r4#show nve vni

r4#show nve peers

r4#show nve inter nve 1 detail

test1.jpg.png

GNS3 Installation on Linux:

https://docs.gns3.com/1QXVIihk7dsOL7Xr7Bmz4zRzTsJ02wklfImGuHwTlaA4/index.html

Cisco CSR1000v appliance:

https://docs.gns3.com/appliances/cisco-csr1000v.html#appliance_supported

Cisco CSR 1000V VxLAN Support:

https://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/vxlan/m_csr-vxlan-support-book.html

VXLAN Basics:

http://stayinginit.blogspot.com/2014/02/vxlan-basics.html

Networking Basics As I Currently Understand

http://stayinginit.blogspot.in/2014/10/vxlan-unicast.html