openstackdvr模式

分布式虚拟路由 Neutron Distributed Virtual Routing（DVR）

前言

​ 为了降低网络节点的负载，同时提高可扩展性，OpenStack 自 Juno 版本开始正式引入了分布式路由（Distributed Virtual Router，DVR）特性（用户可以选择使用与否），来让计算节点自己处理原先的大量东西向流量和非 SNAT 南北流量（有 floating IP 的 vm 跟外面的通信）。

传统（Legacy) router 和DVR router

Legacy router

在Legacy模式下，南北向流量以及不同网络下的东西向流量都要经过网络节点的qrouter，这意味着网络节点将要承受很大的流量和处理负担。

DVR router

在这里插入图片描述

在DVR模式下，计算节点会新起两个namespace:qrouter-ns和fip-ns，绑定浮动ip的南北向流量以及不同网络下的东西向流量将仅在发起通信虚机所在的计算节点上处理，网络节点会新起一个namespace：snat-ns，未绑定浮动ip的南北向流量仍然要经过网络节点。

DVR配置

控制节点

vi  /etc/neutron/neutron.conf
[DEFAULT]
router_distributed = True

vi /etc/neutron/plugins/ml2/ml2_conf.ini 
[ml2]
mechanism_drivers=openvswitch,l2population

重启neutron-server

systemctl restart neutron-server

网络节点

vi /etc/neutron/plugins/ml2/openvswitch_agent.ini 
[agent]
enable_distributed_routing = True
arp_responder  = True
l2_population = True

vi /etc/neutron/l3_agent.ini 
[DEFAULT]
agent_mode = dvr_snat

重启neutron-openvswitch-agent和neutron-l3-agent

systemctl restart neutron-openvswitch-agent
systemctl restart neutron-l3-agent

计算节点

vi /etc/neutron/plugins/ml2/openvswitch_agent.ini 
[agent]
enable_distributed_routing = True
arp_responder  = True
l2_population = True
bridge_mappings=extnet:br-ex

vi /etc/neutron/l3_agent.ini 
[DEFAULT]
interface_driver = openvswitch
agent_mode = dvr

添加br-ex网桥

ovs-vsctl add-br br-ex
ovs-vsctl add-port br-ex ens192（该网卡应能出外网）

修改配置文件

vi /etc/sysconfig/network-scripts/ifcfg-ens256

DEVICE=ens256
NAME=ens256
DEVICETYPE=ovs
TYPE=OVSPort
OVS_BRIDGE=br-ex
ONBOOT=yes
BOOTPROTO=none

vi /etc/sysconfig/network-scripts/ifcfg-br-ex

ONBOOT=yes
PEERDNS=no
NM_CONTROLLED=no
NOZEROCONF=yes
DEVICE=br-ex
NAME=br-ex
DEVICETYPE=ovs
OVSBOOTPROTO=dhcp
TYPE=OVSBridge
OVS_EXTRA="set bridge br-ex fail_mode=standalone"

重启neutron-openvswitch-agent和neutron-l3-agent

systemctl restart neutron-openvswitch-agent
systemctl restart neutron-l3-agent

Network Agent 分布

查看network agent，可以看到计算节点上也会起openvswitch-agent和l3-agent服务。

[root@controller ~(keystone_admin)]# openstack network agent list
+--------------------------------------+--------------------+----------+-------------------+-------+-------+---------------------------+
| ID                                   | Agent Type         | Host     | Availability Zone | Alive | State | Binary                    |
+--------------------------------------+--------------------+----------+-------------------+-------+-------+---------------------------+
| 25784f72-1ebc-4521-936c-ba24c9705395 | Open vSwitch agent | compute1 | nova              | :-)   | UP    | neutron-openvswitch-agent |
| 4ebe6b3c-ffa2-4bd3-b0fd-7cbcf092b5ce | L3 agent           | compute1 | nova              | :-)   | UP    | neutron-l3-agent          |
| 53a6535b-2444-472b-bd95-f3d2ab3e05ab | L3 agent           | compute2 | nova              | :-)   | UP    | neutron-l3-agent          |
| 7079e62a-0e64-47c5-8c48-b04def4abcb9 | Open vSwitch agent | compute2 | nova              | :-)   | UP    | neutron-openvswitch-agent |
| 7e9bea3d-d705-4f02-a272-dd0173cf9ba2 | L3 agent           | network  | nova              | :-)   | UP    | neutron-l3-agent          |
| cad6f5f1-d34a-43c9-bd54-0fd22643e1e8 | Metadata agent     | network  | None              | :-)   | UP    | neutron-metadata-agent    |
| cc0a3672-7f9f-4bff-91a4-25e093555fbd | DHCP agent         | network  | nova              | :-)   | UP    | neutron-dhcp-agent        |
| d4abf92f-edc4-471b-8202-a63dea1f0384 | Open vSwitch agent | network  | nova              | :-)   | UP    | neutron-openvswitch-agent |
+--------------------------------------+--------------------+----------+-------------------+-------+-------+---------------------------+

DVR对2、3层的影响

对3层的影响

计算节点

计算节点会创建两个namespace：qrouter-ns和fip-ns：

[root@compute1 ~]# ip netns list
qrouter-9bfbb054-e777-4c16-9f96-09a16a303576 (id: 0)
fip-b56e71c1-ee9e-4299-b16d-c66951caba21 (id: 1)

qrouter-ns

查看qrouter内部端口，其中：qr口绑定子网网关，rfp口与fip-ns里的fpr口是一对veth pair，用来做router-ns和fip-ns的通信。

[root@compute1 ~]# ip netns exec qrouter-9bfbb054-e777-4c16-9f96-09a16a303576 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: rfp-9bfbb054-e@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether fe:0b:b5:bb:00:16 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 169.254.109.46/31 scope global rfp-9bfbb054-e
       valid_lft forever preferred_lft forever
    inet6 fe80::fc0b:b5ff:febb:16/64 scope link 
       valid_lft forever preferred_lft forever
33: qr-d0f511a4-f5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether fa:16:3e:60:4c:53 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.2/24 brd 192.168.1.255 scope global qr-d0f511a4-f5
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe60:4c53/64 scope link 
       valid_lft forever preferred_lft forever
34: qr-f4b7a3f9-eb: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu