本文记录自己在OVN部署和测试中遇到的问题,以及排查过程
前世今生:
近期在调研OVN,因此就试着搭建了一套openstack/pike版本的OVN。使用OVN作为网络底层实现而替换neutron的各类plugin(L3,DHCP,L2,iptables)
问题出现:
当创建port时,执行neutron port-create [net-id] --name [port-name]命令时,执行失败,log报错如下:
File "/usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 97, in run
txn.results.put(txn.do_commit())
File "/usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 86, in do_commit
command.run_idl(txn)
File "/usr/lib/python2.7/site-packages/networking_ovn/ovsdb/commands.py", line 725, in run_idl
raise RuntimeError(msg)
RuntimeError: Address set as_ip4_b5dbdfe6_bcd3_4e39_b43a_0c37c66ccb54 does not exist. Can't update addresses
错误排查
1、根据报错,我们去看一下commands.py文件中run_idl函数,
根据函数之间的相互调用,我们发现起始调用该代码的地方在ovn-client.py中的create_port函数。
def create_port(self, port):
if utils.is_lsp_ignored(port):
return
port_info = self._get_port_options(port)
external_ids = {ovn_const.OVN_PORT_NAME_EXT_ID_KEY: port['name'],
ovn_const.OVN_DEVID_EXT_ID_KEY: port['device_id'],
ovn_const.OVN_PROJID_EXT_ID_KEY: port['project_id'],
ovn_const.OVN_CIDRS_EXT_ID_KEY: port_info.cidrs}
lswitch_name = utils.ovn_name(port['network_id'])
admin_context = n_context.get_admin_context()
sg_cache = {}
subnet_cache = {}
# It's possible to have a network created on one controller and then a
# port created on a different controller quickly enough that the second
# controller does not yet see that network in its local cache of the
# OVN northbound database. Check if the logical switch is present
# or not in the idl's local copy of the database before creating
# the lswitch port.
self._nb_idl.check_for_row_by_value_and_retry(
'Logical_Switch', 'name', lswitch_name)
with self._nb_idl.transaction(check_error=True) as txn:
if not port_info.dhcpv4_options:
dhcpv4_options = []
elif 'cmd' in port_info.dhcpv4_options:
dhcpv4_options = txn.add(port_info.dhcpv4_options['cmd'])
else:
dhcpv4_options = [port_info.dhcpv4_options['uuid']]
if not port_info.dhcpv6_options:
dhcpv6_options = []
elif 'cmd' in port_info.dhcpv6_options:
dhcpv6_options = txn.add(port_info.dhcpv6_options['cmd'])
else:
dhcpv6_options = [port_info.dhcpv6_options['uuid']]
# The lport_name *must* be neutron port['id']. It must match the
# iface-id set in the Interfaces table of the Open_vSwitch
# database which nova sets to be the port ID.
txn.add(self._nb_idl.create_lswitch_port(
lport_name=port['id'],
lswitch_name=lswitch_name,
addresses=port_info.addresses,
external_ids=external_ids,
parent_name=port_info.parent_name,
tag=port_info.tag,
enabled=port.get('admin_state_up'),
options=port_info.options,
type=port_info.type,
port_security=port_info.port_security,
dhcpv4_options=dhcpv4_options,
dhcpv6_options=dhcpv6_options))
acls_new = ovn_acl.add_acls(self._plugin, admin_context,
port, sg_cache, subnet_cache)
for acl in acls_new:
txn.add(self._nb_idl.add_acl(**acl))
sg_ids = utils.get_lsp_security_groups(port)
if port.get('fixed_ips') and sg_ids:
addresses = ovn_acl.acl_port_ips(port)
# NOTE(rtheis): Fail port creation if the address set doesn't
# exist. This prevents ports from being created on any security
# groups out-of-sync between neutron and OVN.
for sg_id in sg_ids:
for ip_version in addresses:
if addresses[ip_version]:
txn.add(self._nb_idl.update_address_set(
name=utils.ovn_addrset_name(sg_id,
ip_version),
addrs_add=addresses[ip_version],
addrs_remove=None,
if_exists=False))
2、对以上代码进行聚焦,我们发现问题是出在最后一个if语句中的self._nb_idl.update_address_set函数。
3、update_address_set函数调用的是impl_idl_ovn.py中的函数:
/usr/lib/python2.7/site-packages/networking_ovn/ovsdb/impl_idl_ovn.py
4、update_address_set函数调用commands.py中UpdateAddrSetCommand类。
/usr/lib/python2.7/site-packages/networking_ovn/ovsdb/commands.py
5、transaction.py文件中transaction 类的do_commit函数调用run_idl函数:
/usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py
6、run_idl函数是引用commands.py中UpdateAddrSetCommand类中的run_idl函数。
由于我们报错的语句是“RuntimeError: Address set as_ip4_b5dbdfe6_bcd3_4e39_b43a_0c37c66ccb54 does not exist. Can’t update addresses”因此可以认为run_idl函数在实际执行时候,是由于try部分语句执行失败,且if_exists=False,所以才会报错。
当我们试着将ovn-client中涉及到的if_exist的默认值改为True之后,确实不会报错了。
7、现在问题集中到了try部分的语句,该语句引用了idlutils.py函数中的row_by_value函数
/usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py
8、对该函数首先打log,如下:
9、再进行一次port创建
log报错如下:
2019-01-07 09:47:48.996 25329 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=0): AddLSwitchPortCommand(lswitch=neutron-09bb198b-2a90-4585-894b-54f15a21d415, may_exist=True, lport=f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7, columns={'parent_name': [], 'addresses': [u'fa:16:3e:74:d0:1a 20.20.20.7'], 'enabled': True, 'port_security': [u'fa:16:3e:74:d0:1a 20.20.20.7'], 'dhcpv6_options': [], 'external_ids': {'neutron:project_id': u'1349e8d0cb2a41c7bd35c8edc9aef826', 'neutron:cidrs': '20.20.20.7/24', 'neutron:device_id': '', 'neutron:port_name': u'tyx-port-1'}, 'tag': [], 'type': '', 'options': {}, 'dhcpv4_options': [UUID('b3d01780-df1a-4475-b5fb-b8de91a15ed5')]}) do_commit /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:85
2019-01-07 09:47:48.996 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 1111-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:58
2019-01-07 09:47:48.997 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 2222-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:60
2019-01-07 09:47:48.997 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,match=neutron-09bb198b-2a90-4585-894b-54f15a21d415 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:62
2019-01-07 09:47:48.997 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,column=neutron-09bb198b-2a90-4585-894b-54f15a21d415 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:63
2019-01-07 09:47:48.997 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 1111-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:58
2019-01-07 09:47:48.997 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 2222-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:60
2019-01-07 09:47:48.998 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,match=f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:62
2019-01-07 09:47:48.998 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,column=644e0107-fbf0-4dc9-9b61-65861cdf8c55 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:63
2019-01-07 09:47:48.998 25329 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=1): AddACLCommand(lport=f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7, columns={'direction': 'from-lport', 'log': False, 'priority': 1001, 'action': 'drop', 'external_ids': {'neutron:lport': 'f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7'}, 'match': 'inport == "f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7" && ip'}, lswitch=neutron-09bb198b-2a90-4585-894b-54f15a21d415) do_commit /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:85
2019-01-07 09:47:48.999 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 1111-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:58
2019-01-07 09:47:48.999 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 2222-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:60
2019-01-07 09:47:48.999 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,match=neutron-09bb198b-2a90-4585-894b-54f15a21d415 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:62
2019-01-07 09:47:48.999 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,column=neutron-09bb198b-2a90-4585-894b-54f15a21d415 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:63
2019-01-07 09:47:49.000 25329 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=2): AddACLCommand(lport=f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7, columns={'direction': 'to-lport', 'log': False, 'priority': 1001, 'action': 'drop', 'external_ids': {'neutron:lport': 'f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7'}, 'match': 'outport == "f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7" && ip'}, lswitch=neutron-09bb198b-2a90-4585-894b-54f15a21d415) do_commit /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:85
2019-01-07 09:47:49.000 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 1111-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:58
2019-01-07 09:47:49.000 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 2222-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:60
2019-01-07 09:47:49.000 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,match=neutron-09bb198b-2a90-4585-894b-54f15a21d415 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:62
2019-01-07 09:47:49.001 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,column=neutron-09bb198b-2a90-4585-894b-54f15a21d415 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:63
2019-01-07 09:47:49.001 25329 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=3): AddACLCommand(lport=f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7, columns={'direction': 'from-lport', 'log': False, 'priority': 1002, 'action': 'allow', 'external_ids': {'neutron:lport': 'f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7'}, 'match': u'inport == "f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7" && ip4 && ip4.dst == {255.255.255.255, 20.20.20.0/24} && udp && udp.src == 68 && udp.dst == 67'}, lswitch=neutron-09bb198b-2a90-4585-894b-54f15a21d415) do_commit /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:85
2019-01-07 09:47:49.001 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 1111-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:58
2019-01-07 09:47:49.002 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 2222-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:60
2019-01-07 09:47:49.002 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,match=neutron-09bb198b-2a90-4585-894b-54f15a21d415 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:62
2019-01-07 09:47:49.002 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,column=neutron-09bb198b-2a90-4585-894b-54f15a21d415 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:63
2019-01-07 09:47:49.002 25329 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=4): AddACLCommand(lport=f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7, columns={'direction': 'to-lport', 'log': False, 'priority': 1002, 'action': 'allow-related', 'external_ids': {'neutron:lport': 'f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7'}, 'match': u'outport == "f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7" && ip4 && ip4.src == 0.0.0.0/0 && icmp4'}, lswitch=neutron-09bb198b-2a90-4585-894b-54f15a21d415) do_commit /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:85
2019-01-07 09:47:49.003 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 1111-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:58
2019-01-07 09:47:49.003 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 2222-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:60
2019-01-07 09:47:49.003 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,match=neutron-09bb198b-2a90-4585-894b-54f15a21d415 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:62
2019-01-07 09:47:49.003 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,column=neutron-09bb198b-2a90-4585-894b-54f15a21d415 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:63
2019-01-07 09:47:49.004 25329 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=5): AddACLCommand(lport=f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7, columns={'direction': 'from-lport', 'log': False, 'priority': 1002, 'action': 'allow-related', 'external_ids': {'neutron:lport': 'f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7'}, 'match': 'inport == "f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7" && ip6'}, lswitch=neutron-09bb198b-2a90-4585-894b-54f15a21d415) do_commit /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:85
2019-01-07 09:47:49.004 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 1111-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:58
2019-01-07 09:47:49.004 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 2222-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:60
2019-01-07 09:47:49.004 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,match=neutron-09bb198b-2a90-4585-894b-54f15a21d415 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:62
2019-01-07 09:47:49.005 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,column=neutron-09bb198b-2a90-4585-894b-54f15a21d415 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:63
2019-01-07 09:47:49.005 25329 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=6): AddACLCommand(lport=f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7, columns={'direction': 'to-lport', 'log': False, 'priority': 1002, 'action': 'allow-related', 'external_ids': {'neutron:lport': 'f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7'}, 'match': u'outport == "f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7" && ip4 && ip4.src == $as_ip4_b5dbdfe6_bcd3_4e39_b43a_0c37c66ccb54'}, lswitch=neutron-09bb198b-2a90-4585-894b-54f15a21d415) do_commit /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:85
2019-01-07 09:47:49.005 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 1111-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:58
2019-01-07 09:47:49.006 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 2222-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:60
2019-01-07 09:47:49.006 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,match=neutron-09bb198b-2a90-4585-894b-54f15a21d415 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:62
2019-01-07 09:47:49.006 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,column=neutron-09bb198b-2a90-4585-894b-54f15a21d415 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:63
2019-01-07 09:47:49.006 25329 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=7): AddACLCommand(lport=f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7, columns={'direction': 'from-lport', 'log': False, 'priority': 1002, 'action': 'allow-related', 'external_ids': {'neutron:lport': 'f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7'}, 'match': 'inport == "f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7" && ip4'}, lswitch=neutron-09bb198b-2a90-4585-894b-54f15a21d415) do_commit /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:85
2019-01-07 09:47:49.007 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 1111-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:58
2019-01-07 09:47:49.007 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 2222-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:60
2019-01-07 09:47:49.007 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,match=neutron-09bb198b-2a90-4585-894b-54f15a21d415 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:62
2019-01-07 09:47:49.007 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,column=neutron-09bb198b-2a90-4585-894b-54f15a21d415 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:63
2019-01-07 09:47:49.008 25329 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=8): AddACLCommand(lport=f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7, columns={'direction': 'to-lport', 'log': False, 'priority': 1002, 'action': 'allow-related', 'external_ids': {'neutron:lport': 'f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7'}, 'match': u'outport == "f7649ad4-f965-4cfe-82b3-c6e96ec1a2d7" && ip6 && ip6.src == $as_ip6_b5dbdfe6_bcd3_4e39_b43a_0c37c66ccb54'}, lswitch=neutron-09bb198b-2a90-4585-894b-54f15a21d415) do_commit /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:85
2019-01-07 09:47:49.008 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 1111-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:58
2019-01-07 09:47:49.008 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 2222-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:60
2019-01-07 09:47:49.008 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,match=neutron-09bb198b-2a90-4585-894b-54f15a21d415 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:62
2019-01-07 09:47:49.009 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 3333--------------------------------,column=neutron-09bb198b-2a90-4585-894b-54f15a21d415 row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:63
2019-01-07 09:47:49.009 25329 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=9): UpdateAddrSetCommand(if_exists=False, name=as_ip4_b5dbdfe6_bcd3_4e39_b43a_0c37c66ccb54, addrs_remove=None, addrs_add=[u'20.20.20.7']) do_commit /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:85
2019-01-07 09:47:49.009 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 1111-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:58
2019-01-07 09:47:49.010 25329 DEBUG ovsdbapp.backend.ovs_idl.idlutils [-] 2222-------------------------------------- row_by_value /usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py:60
2019-01-07 09:47:49.011 25329 ERROR ovsdbapp.backend.ovs_idl.transaction [req-ba3859fa-4af6-4431-897b-f4c83a568c09 3ebaf54b416a4f04bd1c61600cb67adb 1349e8d0cb2a41c7bd35c8edc9aef826 - default default] Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 97, in run
txn.results.put(txn.do_commit())
File "/usr/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 87, in do_commit
command.run_idl(txn)
File "/usr/lib/python2.7/site-packages/networking_ovn/ovsdb/commands.py", line 725, in run_idl
raise RuntimeError(msg)
RuntimeError: Address set as_ip4_b5dbdfe6_bcd3_4e39_b43a_0c37c66ccb54 does not exist. Can't update addresses
可以看到当idx=9时,也就是for r in tab.rows.values()循环执行到了第10次,无法执行下去,也就是tab.rows.values()的list只有9个值,当去执行第10次循环时,值为空,所以报异常。再回过头看ovn_client.py中的create_port产生这次问题的代码,我们可以发现只有当sg_ids存在时,也就是当安全组存在时,才会去执行下面的条件判断。而错误的log是Address set 不存在,在OVN中一般只有安全组才会用到Address set。
柳暗花明
先看一下ovn中的address_set的表中数据,执行以下命令:
#ovn-nbctl list address_set
结果没有任何输出。
再看看neutron中的安全组信息,执行以下命令,输出结果如下图:
neutron security-group-list
也就是说存在一个default安全组,看到这里,似乎出现了一点眉目,这时我将default安全组删掉,neutron security-group-delete default:
验证猜想
再次执行neutron port-create [net-id] --name [port-name]命令,发现everything goes fine。再分别查看neutron的 sg信息以及ovn的address set信息:
可以发现这两个ID相等。
结论
由于我的部署过程是先将openstack进行完成部署并验证,因此之前的neutron数据库中已经生成了default安全组,当我用OVN替换neutron下层网络功能之后,OVN中并没有该default 安全组的信息,所以导致ovn返回没有找到安全组中绑定的 address set 的信息。