1 组网图
2 驱动管理模式流程
在该模式下，share server虚拟机由manila-share generic驱动管理，包括创建虚拟机、挂盘（nfs server需要的存储盘），创建网络（创建共享网络）。
驱动管理模式需要应用共享网络
2.1 按照generic驱动模式配置manila，启动manila-share。
manila配置文件中，neutron段配置为services租户（manila服务调用，固定在services租户名下）
# cat /etc/manila/manila.conf
[neutron]
url = http://192.168.166.180:9696
www_authenticate_uri = http://192.168.166.180:5000
auth_url = http://192.168.166.180:5000
auth_type = password
project_domain_name = Default
user_domain_name = Default
region_name = RegionOne
project_name = services //services租户
username = neutron
password = neutron
manila-share启动后，manila调用neutron接口在services租户名下创建manila共享服务网络，共享服务网络名固定为manila_service_network。初始创建的共享服务网络不带子网。
子网在创建共享网络时，自动创建。
# openstack network list
+--------------------------------------+------------------------+--------------------------------------+
| ID | Name | Subnets |
+--------------------------------------+------------------------+--------------------------------------+
| 04cb080f-3290-46c3-82a2-3b24bdcfbab0 | manila_service_network | |
| 3c83fdd8-c1d6-441b-8163-84d5daa18931 | ext-net | 778fa9ea-2ef6-463b-894a-f456e90399aa |
| 9ee4e64a-da83-4cfd-aded-f6542bbd3937 | net_api | cf87d2b1-7fb3-4f90-996a-c7600ed0f27d |
+--------------------------------------+------------------------+--------------------------------------+
配置使用ovs插件时，会在br-int桥上创建一个tap口，关联到manila_service_network
19: tap512369b0-c9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1442 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether fa:16:3e:9c:17:f7 brd ff:ff:ff:ff:ff:ff
inet6 fe80::f816:3eff:fe9c:17f7/64 scope link
valid_lft forever preferred_lft forever
[root@openstack4k8s work(keystone_admin)]# ovs-vsctl show
0e0010bb-71a5-4222-a29c-adb95e075504
Manager "ptcp:6640:127.0.0.1"
is_connected: true
Bridge br-int
fail_mode: secure
datapath_type: system
Port "tap512369b0-c9"
Interface "tap512369b0-c9"
type: internal
Port "patch-br-int-to-provnet-3c83fdd8-c1d6-441b-8163-84d5daa18931"
Interface "patch-br-int-to-provnet-3c83fdd8-c1d6-441b-8163-84d5daa18931"
type: patch
options: {peer="patch-provnet-3c83fdd8-c1d6-441b-8163-84d5daa18931-to-br-int"}
Port "tapb3c7059d-60"
Interface "tapb3c7059d-60"
Port "tap9d17b84a-a9"
Interface "tap9d17b84a-a9"
Port br-int
Interface br-int
type: internal
Port "tapc33e597a-1d"
Interface "tapc33e597a-1d"
Port "tapb5bb6ae1-f4"
Interface "tapb5bb6ae1-f4"
Port "tap850b6f91-2b"
Interface "tap850b6f91-2b"
Bridge br-ex
Port br-ex
Interface br-ex
type: internal
Port "patch-provnet-3c83fdd8-c1d6-441b-8163-84d5daa18931-to-br-int"
Interface "patch-provnet-3c83fdd8-c1d6-441b-8163-84d5daa18931-to-br-int"
type: patch
options: {peer="patch-br-int-to-provnet-3c83fdd8-c1d6-441b-8163-84d5daa18931"}
ovs_version: "2.12.0"
2.2 OpenStack集群中需要创建一个router
# openstack router list
+--------------------------------------+--------+--------+-------+----------------------------------+
| ID | Name | Status | State | Project |
+--------------------------------------+--------+--------+-------+----------------------------------+
| cc5fb732-045d-4a9f-9d46-d156a23a7d50 | router | ACTIVE | UP | 23a40789342d4de6bc06d0b0bef7069c |
+--------------------------------------+--------+--------+-------+----------------------------------+
在创建共享网络之前，租户网络必须先挂到路由器上，不然会创建失败
[root@openstack4k8s ~(keystone_demo)]# tail -f /var/log/manila/share.log
2020-05-19 13:03:28.265 128182 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/share/drivers/service_instance.py", line 9 02, in setup_network
2020-05-19 13:03:28.265 128182 ERROR oslo_messaging.rpc.server neutron_net_id, neutron_subnet_id)
2020-05-19 13:03:28.265 128182 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 328, in inne r
2020-05-19 13:03:28.265 128182 ERROR oslo_messaging.rpc.server return f(*args, **kwargs)
2020-05-19 13:03:28.265 128182 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/manila/share/drivers/service_instance.py", line 1 072, in _get_private_router
2020-05-19 13:03:28.265 128182 ERROR oslo_messaging.rpc.server _('Subnet gateway is not attached to the router.'))
2020-05-19 13:03:28.265 128182 ERROR oslo_messaging.rpc.server ServiceInstanceException: Subnet gateway is not attached to the router.
2020-05-19 13:03:28.265 128182 ERROR oslo_messaging.rpc.server
2020-05-19 13:03:30.134 128181 INFO manila.share.manager [req-75f80a61-41c3-410c-8784-3345b48f3bad - - - - -] Updating share status
2020-05-19 13:03:46.083 128182 INFO manila.share.manager [req-5ae35dda-de5f-4e3a-a1cd-5d2dbbdc26d8 - - - - -] Updating share status
2.3 创建共享网络
使用net_api这个租户网络创建共享网络
首先创建port，并挂到router上
[root@openstack4k8s k8s(keystone_services)]# openstack network list
+--------------------------------------+------------------------+----------------------------------------------------------------------------+
| ID | Name | Subnets |
+--------------------------------------+------------------------+--------------------------------------------------------------------------
| 04cb080f-3290-46c3-82a2-3b24bdcfbab0 | manila_service_network | 71899bf2-c792-48e3-ba7b-a556110e372a, ff38680c-6a3d-4be9-8e86-cf29b0082942 |
| 3c83fdd8-c1d6-441b-8163-84d5daa18931 | ext-net | 778fa9ea-2ef6-463b-894a-f456e90399aa |
| 9ee4e64a-da83-4cfd-aded-f6542bbd3937 | net_api | cf87d2b1-7fb3-4f90-996a-c7600ed0f27d |
| a9d09889-e9f5-4b6b-aba0-bcf46959300f | net_app | bd9c5c74-a4ac-4859-a480-aadbd213bb7e |
+--------------------------------------+------------------------+----------------------------------------------------------------------------+
# openstack port list |grep cf87d2b1-7fb3-4f90-996a-c7600ed0f27d
| 19167e0e-8fa4-4697-ae87-9aefac3cc7b3 | | fa:16:3e:a0:5c:89 | ip_address='192.168.1.1', subnet_id='cf87d2b1-7fb3-4f90-996a-c7600ed0f27d' | ACTIVE |
# openstack port show 19167e0e-8fa4-4697-ae87-9aefac3cc7b3
+-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | UP |
| allowed_address_pairs | |
| binding_host_id | |
| binding_profile | |
| binding_vif_details | |
| binding_vif_type | unbound |
| binding_vnic_type | normal |
| created_at | 2020-03-17T17:21:52Z |
| data_plane_status | None |
| description | |
| device_id | cc5fb732-045d-4a9f-9d46-d156a23a7d50 |
| device_owner | network:router_interface |
| dns_assignment | None |
| dns_domain | None |
| dns_name | None |
| extra_dhcp_opts | |
| fixed_ips | ip_address='192.168.1.1', subnet_id='cf87d2b1-7fb3-4f90-996a-c7600ed0f27d' |
| id | 19167e0e-8fa4-4697-ae87-9aefac3cc7b3 |
| location | cloud='', project.domain_id=, project.domain_name=, project.id='23a40789342d4de6bc06d0b0bef7069c', project.name=, region_name='RegionOne', zone= |
| mac_address | fa:16:3e:a0:5c:89 |
| name | |
| network_id | 9ee4e64a-da83-4cfd-aded-f6542bbd3937 |
| port_security_enabled | False |
| project_id | 23a40789342d4de6bc06d0b0bef7069c |
| propagate_uplink_status | None |
| qos_policy_id | None |
| resource_request | None |
| revision_number | 3 |
| security_group_ids | |
| status | ACTIVE |
| tags | |
| trunk_details | None |
| updated_at | 2020-03-17T17:21:53Z |
+-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+
执行manila命令创建共享网络
# manila share-network-create --name demo-share-network1 --neutron-net-id 9ee4e64a-da83-4cfd-aded-f6542bbd3937 --neutron-subnet-id cf87d2b1-7fb3-4f90-996a-c7600ed0f27d
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------+
| Property | Value |
+-----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------+
| description | None |
| name | demo-share-network1 |
| share_network_subnets | [{u'availability_zone': None, u'neutron_net_id': u'9ee4e64a-da83-4cfd-aded-f6542bbd3937', u'created_at': u'2020-05-18T19:08:40.000000', u'neutron_subnet_id': u'cf87d2b1-7fb3-4f90-996a-c7600ed0f27d', u'updated_at': None, u'mtu': None, u'id': u'7fba0be0-e7b0-4147-beb7-427cff9b8866' , u'segmentation_id': None, u'ip_version': None, u'cidr': None, u'network_type': None, u'gateway': None}] |
| created_at | 2020-05-18T19:08:40.000000 |
| project_id | 8e9f4288a98b46cc881b9e8bcc1659f5 |
| updated_at | None |
| id | 80ee729c-a454-4760-9928-2601af555b98 |
+-----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------+
manila-share会给manila_service_network创建一个子网，并创建一个port，挂到router上。该子网默认13个地址，掩码为28位
# openstack network list
+--------------------------------------+------------------------+--------------------------------------+
| ID | Name | Subnets |
+--------------------------------------+------------------------+--------------------------------------+
| 04cb080f-3290-46c3-82a2-3b24bdcfbab0 | manila_service_network | ff38680c-6a3d-4be9-8e86-cf29b0082942 |
| 3c83fdd8-c1d6-441b-8163-84d5daa18931 | ext-net | 778fa9ea-2ef6-463b-894a-f456e90399aa |
| 9ee4e64a-da83-4cfd-aded-f6542bbd3937 | net_api | cf87d2b1-7fb3-4f90-996a-c7600ed0f27d |
+--------------------------------------+------------------------+--------------------------------------+
# openstack subnet show ff38680c-6a3d-4be9-8e86-cf29b0082942
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| allocation_pools | 10.254.0.2-10.254.0.14 |
| cidr | 10.254.0.0/28 |
| created_at | 2020-05-18T19:13:12Z |
| description | |
| dns_nameservers | |
| enable_dhcp | True |
| gateway_ip | 10.254.0.1 |
| host_routes | |
| id | ff38680c-6a3d-4be9-8e86-cf29b0082942 |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| location | cloud='', project.domain_id=, project.domain_name='Default', project.id='99671b33717d4be5a6ee2227c39da017', project.name='services', region_name='RegionOne', zone= |
| name | service_subnet_for_handling_of_share_server_for_tenant_subnet_cf87d2b1-7fb3-4f90-996a-c7600ed0f27d |
| network_id | 04cb080f-3290-46c3-82a2-3b24bdcfbab0 |
| prefix_length | None |
| project_id | 99671b33717d4be5a6ee2227c39da017 |
| revision_number | 0 |
| segment_id | None |
| service_types | |
| subnetpool_id | None |
| tags | |
| updated_at | 2020-05-18T19:13:12Z |
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
子网的第一个地址10.254.0.1分给了router的port
[root@openstack4k8s k8s(keystone_services)]# openstack port list|grep ff38680c-6a3d-4be9-8e86-cf29b0082942
| 512369b0-c969-4760-bebd-de188b7282e5 | | fa:16:3e:9c:17:f7 | ip_address='10.254.0.10', subnet_id='ff38680c-6a3d-4be9-8e86-cf29b0082942' | ACTIVE |
| 6a68543a-cf43-490e-be2c-cc59e8849690 | | fa:16:3e:80:73:75 | ip_address='10.254.0.1', subnet_id='ff38680c-6a3d-4be9-8e86-cf29b0082942' | ACTIVE |
| | | | ip_address='10.254.0.2', subnet_id='ff38680c-6a3d-4be9-8e86-cf29b0082942' | |
| 9edbafc1-f43a-4f0b-ab77-3c4ef273fa8a | | fa:16:3e:90:51:f8 | ip_address='10.254.0.8', subnet_id='ff38680c-6a3d-4be9-8e86-cf29b0082942' | ACTIVE |
[root@openstack4k8s k8s(keystone_services)]#
子网的另外一个port给了manila_service_network的tap口，在tap口上增加ip地址10.254.0.10，通过这个地址，manila-share可以操作后面创建的share server虚拟机
19: tap512369b0-c9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1442 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether fa:16:3e:9c:17:f7 brd ff:ff:ff:ff:ff:ff
inet 10.254.0.10/28 brd 10.254.0.15 scope global tap512369b0-c9
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe9c:17f7/64 scope link
valid_lft forever preferred_lft forever
在demo租户空间查看share network列表
[root@openstack4k8s work(keystone_demo)]# manila share-network-list
+--------------------------------------+---------------------+
| id | name |
+--------------------------------------+---------------------+
| 80ee729c-a454-4760-9928-2601af555b98 | demo-share-network1 |
+--------------------------------------+---------------------+
2.4 创建share，共享文件系统
调用manila接口创建share，需要指定使用哪个共享网络和share type
[root@openstack4k8s ~(keystone_demo)]# manila create NFS 1 --name demo-share1 --share-network demo-share-network1 --share-type generic_share_type
+---------------------------------------+--------------------------------------+
| Property | Value |
+---------------------------------------+--------------------------------------+
| status | creating |
| share_type_name | generic_share_type |
| description | None |
| availability_zone | None |
| share_network_id | 80ee729c-a454-4760-9928-2601af555b98 |
| share_group_id | None |
| revert_to_snapshot_support | False |
| access_rules_status | active |
| snapshot_id | None |
| create_share_from_snapshot_support | False |
| is_public | False |
| task_state | None |
| snapshot_support | False |
| id | 172e3333-8960-4599-9f43-f8edc2156143 |
| size | 1 |
| source_share_group_snapshot_member_id | None |
| user_id | cdd65fbe8be24b3fba5bf0ff0f14bc7c |
| name | demo-share1 |
| share_type | b815b012-29ad-4644-bc7b-2058959b3d23 |
| has_replicas | False |
| replication_type | None |
| created_at | 2020-05-18T19:13:09.000000 |
| share_proto | NFS |
| mount_snapshot_support | False |
| project_id | 8e9f4288a98b46cc881b9e8bcc1659f5 |
| metadata | {} |
+---------------------------------------+--------------------------------------+
[root@openstack4k8s ~(keystone_demo)]# manila list
+--------------------------------------+-------------+------+-------------+----------+-----------+--------------------+------+-------------------+
| ID | Name | Size | Share Proto | Status | Is Public | Share Type Name | Host | Availability Zone |
+--------------------------------------+-------------+------+-------------+----------+-----------+--------------------+------+-------------------+
| 172e3333-8960-4599-9f43-f8edc2156143 | demo-share1 | 1 | NFS | creating | False | generic_share_type | | nova |
+--------------------------------------+-------------+------+-------------+----------+-----------+--------------------+------+-------------------+
[root@openstack4k8s ~(keystone_demo)]# manila list
+--------------------------------------+-------------+------+-------------+-----------+-----------+--------------------+------+-------------------+
| ID | Name | Size | Share Proto | Status | Is Public | Share Type Name | Host | Availability Zone |
+--------------------------------------+-------------+------+-------------+-----------+-----------+--------------------+------+-------------------+
| 172e3333-8960-4599-9f43-f8edc2156143 | demo-share1 | 1 | NFS | available | False | generic_share_type | | nova |
+--------------------------------------+-------------+------+-------------+-----------+-----------+--------------------+------+-------------------+
日志
[root@openstack4k8s ~(keystone_demo)]# tail -f /var/log/manila/share.log
2020-05-18 15:10:30.021 128181 INFO manila.share.manager [req-75f80a61-41c3-410c-8784-3345b48f3bad - - - - -] Updating share status
2020-05-18 15:10:45.514 128182 INFO manila.share.manager [req-5ae35dda-de5f-4e3a-a1cd-5d2dbbdc26d8 - - - - -] Check for unused share servers to delete.
2020-05-18 15:10:45.561 128182 INFO manila.share.manager [req-5ae35dda-de5f-4e3a-a1cd-5d2dbbdc26d8 - - - - -] Updating share status
2020-05-18 15:11:30.010 128181 INFO manila.share.manager [req-75f80a61-41c3-410c-8784-3345b48f3bad - - - - -] Updating share status
2020-05-18 15:11:45.521 128182 INFO manila.share.manager [req-5ae35dda-de5f-4e3a-a1cd-5d2dbbdc26d8 - - - - -] Updating share status
2020-05-18 15:12:30.001 128181 INFO manila.share.manager [req-75f80a61-41c3-410c-8784-3345b48f3bad - - - - -] Updating share status
2020-05-18 15:12:45.518 128182 INFO manila.share.manager [req-5ae35dda-de5f-4e3a-a1cd-5d2dbbdc26d8 - - - - -] Updating share status
2020-05-18 15:13:11.614 128182 WARNING manila.share.drivers.service_instance [req-6cac756b-dcfe-4de1-b649-8e92df063c6f cdd65fbe8be24b3fba5bf0ff0f14bc7c 8 e9f4288a98b46cc881b9e8bcc1659f5 - - -] No key path is available. May be non-existent key path is provided. Check path_to_private_key (current value None) and path_to_public_key (current value ~/.ssh/id_rsa.pub) in manila configuration file.: ShareServerNotFoundByFilters: Share server could not be found by filters: share_network_subnet_id is "7fba0be0-e7b0-4147-beb7-427cff9b8866", host is "openstack4k8s@generic" and status in "creating" or "active".
2020-05-18 15:13:17.446 128182 INFO manila.network.linux.interface [req-6cac756b-dcfe-4de1-b649-8e92df063c6f cdd65fbe8be24b3fba5bf0ff0f14bc7c 8e9f4288a98 b46cc881b9e8bcc1659f5 - - -] Device tap512369b0-c9 already exists.
2020-05-18 15:13:30.006 128181 INFO manila.share.manager [req-75f80a61-41c3-410c-8784-3345b48f3bad - - - - -] Updating share status
2020-05-18 15:13:45.522 128182 INFO manila.share.manager [req-5ae35dda-de5f-4e3a-a1cd-5d2dbbdc26d8 - - - - -] Updating share status
manila-share在services租户空间创建一个虚拟机instance-00000011
[root@openstack4k8s ~(keystone_demo)]# virsh list
Id Name State
----------------------------------------------------
1 instance-0000000e running
2 instance-0000000f running
3 instance-0000000d running
4 instance-00000010 running
[root@openstack4k8s ~(keystone_demo)]# virsh list
Id Name State
----------------------------------------------------
1 instance-0000000e running
2 instance-0000000f running
3 instance-0000000d running
4 instance-00000010 running
5 instance-00000011 running
[root@openstack4k8s k8s(keystone_services)]# openstack server list
+--------------------------------------+--------------------------------------+--------+------------------------------------+----------------------+--------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+--------------------------------------+--------+------------------------------------+----------------------+--------+
| 1e4a7b73-df15-4c03-888f-df27df4cd435 | 823fa69a-b9cb-4f72-a72b-43e45fccb3ec | ACTIVE | manila_service_network=10.254.0.8 | manila-service-image | k8s |
+--------------------------------------+--------------------------------------+--------+------------------------------------+----------------------+--------+
manila-share调用neutron接口在manila_service_network的子网ff38680c-6a3d-4be9-8e86-cf29b0082942上创建一个port，挂到该虚拟机上，分配的地址为10.254.0.8
此port在services租户空间
[root@openstack4k8s k8s(keystone_services)]# openstack port list |grep ff38680c-6a3d-4be9-8e86-cf29b0082942
| 512369b0-c969-4760-bebd-de188b7282e5 | | fa:16:3e:9c:17:f7 | ip_address='10.254.0.10', subnet_id='ff38680c-6a3d-4be9-8e86-cf29b0082942' | ACTIVE |
| 6a68543a-cf43-490e-be2c-cc59e8849690 | | fa:16:3e:80:73:75 | ip_address='10.254.0.1', subnet_id='ff38680c-6a3d-4be9-8e86-cf29b0082942' | ACTIVE |
| | | | ip_address='10.254.0.2', subnet_id='ff38680c-6a3d-4be9-8e86-cf29b0082942' | |
| 9edbafc1-f43a-4f0b-ab77-3c4ef273fa8a | | fa:16:3e:90:51:f8 | ip_address='10.254.0.8', subnet_id='ff38680c-6a3d-4be9-8e86-cf29b0082942' | ACTIVE |
manila-share调用cinder接口在services租户空间创建一个卷，并挂到该虚拟机上
[root@openstack4k8s k8s(keystone_services)]# openstack volume list
+--------------------------------------+---------------------------------------------------+--------+------+---------------------------------------------------------------+
| ID | Name | Status | Size | Attached to |
+--------------------------------------+---------------------------------------------------+--------+------+---------------------------------------------------------------+
| 4a6f4fac-ebf5-4d26-81d8-a67b160c63bc | manila-share-bb80d66f-2764-465b-a5bf-ff0ca6047375 | in-use | 1 | Attached to 823fa69a-b9cb-4f72-a72b-43e45fccb3ec on /dev/vdb |
+--------------------------------------+---------------------------------------------------+--------+------+---------------------------------------------------------------+
查看share，path可以直接使用
[root@openstack4k8s ~(keystone_demo)]# manila show demo-share1
+---------------------------------------+----------------------------------------------------------------------+
| Property | Value |
+---------------------------------------+----------------------------------------------------------------------+
| status | available |
| share_type_name | generic_share_type |
| description | None |
| availability_zone | nova |
| share_network_id | 80ee729c-a454-4760-9928-2601af555b98 |
| export_locations | |
| | path = 10.254.0.8:/shares/share-bb80d66f-2764-465b-a5bf-ff0ca6047375 |
| | id = 52a88bec-5fe4-483d-af72-b2117d0eb2a8 |
| | preferred = False |
| share_group_id | None |
| revert_to_snapshot_support | False |
| access_rules_status | active |
| snapshot_id | None |
| create_share_from_snapshot_support | False |
| is_public | False |
| task_state | None |
| snapshot_support | False |
| id | 172e3333-8960-4599-9f43-f8edc2156143 |
| size | 1 |
| source_share_group_snapshot_member_id | None |
| user_id | cdd65fbe8be24b3fba5bf0ff0f14bc7c |
| name | demo-share1 |
| share_type | b815b012-29ad-4644-bc7b-2058959b3d23 |
| has_replicas | False |
| replication_type | None |
| created_at | 2020-05-18T19:13:09.000000 |
| share_proto | NFS |
| mount_snapshot_support | False |
| project_id | 8e9f4288a98b46cc881b9e8bcc1659f5 |
| metadata | {} |
+---------------------------------------+----------------------------------------------------------------------+
2.5 使用share
创建access，运行k8s-m2虚拟机使用这个share，k8s-m2虚拟机挂在net-api这个租户网络上，分到的ip地址为192.168.1.243
[root@openstack4k8s ~(keystone_demo)]# manila access-allow demo-share1 ip 192.168.1.243
+--------------+--------------------------------------+
| Property | Value |
+--------------+--------------------------------------+
| access_key | None |
| share_id | 172e3333-8960-4599-9f43-f8edc2156143 |
| created_at | 2020-05-19T09:32:54.000000 |
| updated_at | None |
| access_type | ip |
| access_to | 192.168.1.243 |
| access_level | rw |
| state | queued_to_apply |
| id | 2df3953c-6b70-4ea0-b8e1-498748fabbea |
| metadata | {} |
+--------------+--------------------------------------+
[root@openstack4k8s ~(keystone_demo)]# manila access-list demo-share1
+--------------------------------------+-------------+---------------+--------------+--------+------------+----------------------------+------------+
| id | access_type | access_to | access_level | state | access_key | created_at | updated_at |
+--------------------------------------+-------------+---------------+--------------+--------+------------+----------------------------+------------+
| 2df3953c-6b70-4ea0-b8e1-498748fabbea | ip | 192.168.1.243 | rw | active | None | 2020-05-19T09:32:54.000000 | None |
+--------------------------------------+-------------+---------------+--------------+--------+------------+----------------------------+------------+
使用manila/manila进入manila虚拟机，查看nfs文件系统已经export
[root@openstack4k8s ~]# ssh manila@10.254.0.8
manila@10.254.0.8's password:
Welcome to Ubuntu 18.04.4 LTS (GNU/Linux 4.15.0-96-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
Last login: Tue May 19 09:15:14 2020 from 10.254.0.10
manila@ubuntu:~$ sudo cat /etc/exports
# /etc/exports: the access control list for filesystems which may be exported
# to NFS clients. See exports(5).
#
# Example for NFSv2 and NFSv3:
# /srv/homes hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
#
# Example for NFSv4:
# /srv/nfs4 gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes gss/krb5i(rw,sync,no_subtree_check)
#
manila@ubuntu:~$ df
Filesystem 1K-blocks Used Available Use% Mounted on
udev 4061596 0 4061596 0% /dev
tmpfs 816760 2000 814760 1% /run
/dev/vda1 1913908 1127712 664232 63% /
tmpfs 4083784 0 4083784 0% /dev/shm
tmpfs 5120 0 5120 0% /run/lock
tmpfs 4083784 0 4083784 0% /sys/fs/cgroup
tmpfs 816756 0 816756 0% /run/user/1000
/dev/vdb 999320 2564 927944 1% /shares/share-bb80d66f-2764-465b-a5bf-ff0ca6047375
manila@ubuntu:~$ sudo fdisk -l
Disk /dev/vda: 2 GiB, 2164391936 bytes, 4227328 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x120ed681
Device Boot Start End Sectors Size Id Type
/dev/vda1 * 2048 4225279 4223232 2G 83 Linux
Disk /dev/vdb: 1 GiB, 1073741824 bytes, 2097152 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
manila@ubuntu:~$ sudo ls /shares/share-bb80d66f-2764-465b-a5bf-ff0ca6047375/
lost+found
manila@ubuntu:~$ sudo cat /etc/exports
/shares/share-bb80d66f-2764-465b-a5bf-ff0ca6047375 192.168.1.243(rw,sync,wdelay,hide,nocrossmnt,secure,no_root_squash,no_all_squash,no_subtree_check ,secure_locks,acl,no_pnfs,anonuid=65534,anongid=65534,sec=sys,rw,secure,no_root_squash,no_all_squash)
manila@ubuntu:~$ cat /shares/share-bb80d66f-2764-465b-a5bf-ff0ca6047375/test
hello manila with share server
manila@ubuntu:~$ sudo exportfs -v
/shares/share-bb80d66f-2764-465b-a5bf-ff0ca6047375
192.168.1.243(rw,wdelay,no_root_squash,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash)
manila@ubuntu:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1442 qdisc fq_codel state UP group default qlen 1000
link/ether fa:16:3e:90:51:f8 brd ff:ff:ff:ff:ff:ff
inet 10.254.0.8/28 brd 10.254.0.15 scope global ens3
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe90:51f8/64 scope link
valid_lft forever preferred_lft forever
在k8s-m2这个虚拟机中，挂载并使用这个share
[root@k8s-m2 ~]# mount -vt nfs 10.254.0.8:/shares/share-bb80d66f-2764-465b-a5bf-ff0ca6047375 /home/manila_mnt/
mount.nfs: timeout set for Tue May 19 05:36:21 2020
mount.nfs: trying text-based options 'vers=4.1,addr=10.254.0.8,clientaddr=192.168.1.243'
[root@k8s-m2 ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
10.254.0.8:/shares/share-bb80d66f-2764-465b-a5bf-ff0ca6047375 999424 2048 928768 1% /home/manila_mnt
[root@k8s-m2 ~]# touch /home/manila_mnt/test
[root@k8s-m2 ~]# echo "hello manila with share server" > /home/manila_mnt/test
[root@k8s-m2 ~]# exit
文件已经写到manila虚拟机的nfs server上
[root@openstack4k8s k8s(keystone_services)]# ssh manila@10.254.0.8
manila@10.254.0.8's password:
Welcome to Ubuntu 18.04.4 LTS (GNU/Linux 4.15.0-96-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
Last login: Tue May 19 09:32:05 2020 from 10.254.0.10
manila@ubuntu:~$ df
Filesystem 1K-blocks Used Available Use% Mounted on
udev 4061596 0 4061596 0% /dev
tmpfs 816760 2000 814760 1% /run
/dev/vda1 1913908 1127720 664224 63% /
tmpfs 4083784 0 4083784 0% /dev/shm
tmpfs 5120 0 5120 0% /run/lock
tmpfs 4083784 0 4083784 0% /sys/fs/cgroup
tmpfs 816756 0 816756 0% /run/user/1000
/dev/vdb 999320 2568 927940 1% /shares/share-bb80d66f-2764-465b-a5bf-ff0ca6047375
manila@ubuntu:~$ cat /shares/share-bb80d66f-2764-465b-a5bf-ff0ca6047375/
lost+found/ test
manila@ubuntu:~$ cat /shares/share-bb80d66f-2764-465b-a5bf-ff0ca6047375/test
hello manila with share server
2.6 创建第二个共享网络
创建租户网络net-app
[root@openstack4k8s k8s(keystone_services)]# openstack network list
+--------------------------------------+------------------------+----------------------------------------------------------------------------+
| ID | Name | Subnets |
+--------------------------------------+------------------------+----------------------------------------------------------------------------+
| 04cb080f-3290-46c3-82a2-3b24bdcfbab0 | manila_service_network | 71899bf2-c792-48e3-ba7b-a556110e372a, ff38680c-6a3d-4be9-8e86-cf29b0082942 |
| 3c83fdd8-c1d6-441b-8163-84d5daa18931 | ext-net | 778fa9ea-2ef6-463b-894a-f456e90399aa |
| 9ee4e64a-da83-4cfd-aded-f6542bbd3937 | net_api | cf87d2b1-7fb3-4f90-996a-c7600ed0f27d |
| a9d09889-e9f5-4b6b-aba0-bcf46959300f | net_app | bd9c5c74-a4ac-4859-a480-aadbd213bb7e |
+--------------------------------------+------------------------+----------------------------------------------------------------------------+
net-app租户网络，子网需要创建一个port挂到router上，分的地址为192.168.2.1
[root@openstack4k8s k8s(keystone_services)]# openstack port list | grep bd9c5c74-a4ac-4859-a480-aadbd213bb7e
| 3e867a13-6aeb-492d-a25e-4914831062f5 | | fa:16:3e:4d:9b:07 | ip_address='192.168.2.1', subnet_id='bd9c5c74-a4ac-4859-a480-aadbd213bb7e' | ACTIVE |
| 6141ed86-b83f-4e4e-a56f-1dac2008de09 | | fa:16:3e:4e:35:de | ip_address='192.168.2.2', subnet_id='bd9c5c74-a4ac-4859-a480-aadbd213bb7e' | DOWN |
[root@openstack4k8s k8s(keystone_services)]#
创建共享网络，会在manila_service_network上创建一个子网 10.254.0.16/28
[root@openstack4k8s k8s(keystone_services)]# openstack subnet list
+--------------------------------------+----------------------------------------------------------------------------------------------------+--------------------------------------+------------------+
| ID | Name | Network | Subnet |
+--------------------------------------+----------------------------------------------------------------------------------------------------+--------------------------------------+------------------+
| 71899bf2-c792-48e3-ba7b-a556110e372a | service_subnet_for_handling_of_share_server_for_tenant_subnet_bd9c5c74-a4ac-4859-a480-aadbd213bb7e | 04cb080f-3290-46c3-82a2-3b24bdcfbab0 | 10.254.0.16/28 |
| 778fa9ea-2ef6-463b-894a-f456e90399aa | ext-subnet | 3c83fdd8-c1d6-441b-8163-84d5daa18931 | 192.168.100.0/24 |
| bd9c5c74-a4ac-4859-a480-aadbd213bb7e | subnet_app | a9d09889-e9f5-4b6b-aba0-bcf46959300f | 192.168.2.0/24 |
| cf87d2b1-7fb3-4f90-996a-c7600ed0f27d | subnet_api | 9ee4e64a-da83-4cfd-aded-f6542bbd3937 | 192.168.1.0/24 |
| ff38680c-6a3d-4be9-8e86-cf29b0082942 | service_subnet_for_handling_of_share_server_for_tenant_subnet_cf87d2b1-7fb3-4f90-996a-c7600ed0f27d | 04cb080f-3290-46c3-82a2-3b24bdcfbab0 | 10.254.0.0/28 |
+--------------------------------------+----------------------------------------------------------------------------------------------------+--------------------------------
基于这个子网创建一个port挂到tap口上（同一个port 两个地址）
[root@openstack4k8s k8s(keystone_services)]# openstack port list
| 512369b0-c969-4760-bebd-de188b7282e5 | | fa:16:3e:9c:17:f7 | ip_address='10.254.0.10', subnet_id='ff38680c-6a3d-4be9-8e86-cf29b0082942' | ACTIVE |
| | | | ip_address='10.254.0.22', subnet_id='71899bf2-c792-48e3-ba7b-a556110e372a' | |
19: tap512369b0-c9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1442 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether fa:16:3e:9c:17:f7 brd ff:ff:ff:ff:ff:ff
inet 10.254.0.10/28 brd 10.254.0.15 scope global tap512369b0-c9
valid_lft forever preferred_lft forever
inet 10.254.0.22/28 brd 10.254.0.31 scope global tap512369b0-c9
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe9c:17f7/64 scope link
valid_lft forever preferred_lft forever
创建第二个共享网络虚拟机
# openstack server list
+--------------------------------------+--------------------------------------+--------+------------------------------------+----------------------+--------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+--------------------------------------+--------+------------------------------------+----------------------+--------+
| ccca9196-d126-4416-9c79-50e6f32806db | 55882b22-a1ca-450a-8b90-8e44876360e0 | ACTIVE | manila_service_network=10.254.0.23 | manila-service-image | k8s |
| 1e4a7b73-df15-4c03-888f-df27df4cd435 | 823fa69a-b9cb-4f72-a72b-43e45fccb3ec | ACTIVE | manila_service_network=10.254.0.8 | manila-service-image | k8s |
+--------------------------------------+--------------------------------------+--------+------------------------------------+----------------------+--------+
基于第二个共享网络创建多个share，manila-share发现共享网络对应的虚机已经存在就不会再创建新的虚拟机，使用已经存在的虚拟机
[root@openstack4k8s images(keystone_admin)]# tail -f /var/log/manila/share.log
2020-05-19 13:25:46.084 128182 INFO manila.share.manager [req-5ae35dda-de5f-4e3a-a1cd-5d2dbbdc26d8 - - - - -] Updating share status
2020-05-19 13:26:30.146 128181 INFO manila.share.manager [req-75f80a61-41c3-410c-8784-3345b48f3bad - - - - -] Updating share status
2020-05-19 13:26:46.086 128182 INFO manila.share.manager [req-5ae35dda-de5f-4e3a-a1cd-5d2dbbdc26d8 - - - - -] Updating share status
2020-05-19 13:27:30.155 128181 INFO manila.share.manager [req-75f80a61-41c3-410c-8784-3345b48f3bad - - - - -] Updating share status
2020-05-19 13:27:46.091 128182 INFO manila.share.manager [req-5ae35dda-de5f-4e3a-a1cd-5d2dbbdc26d8 - - - - -] Updating share status
2020-05-19 13:28:30.140 128181 INFO manila.share.manager [req-75f80a61-41c3-410c-8784-3345b48f3bad - - - - -] Updating share status
2020-05-19 13:28:46.099 128182 INFO manila.share.manager [req-5ae35dda-de5f-4e3a-a1cd-5d2dbbdc26d8 - - - - -] Updating share status
2020-05-19 13:29:30.142 128181 INFO manila.share.manager [req-75f80a61-41c3-410c-8784-3345b48f3bad - - - - -] Updating share status
2020-05-19 13:29:46.065 128182 INFO manila.share.manager [req-5ae35dda-de5f-4e3a-a1cd-5d2dbbdc26d8 - - - - -] Check for unused share servers to delete.
2020-05-19 13:29:46.099 128182 INFO manila.share.manager [req-5ae35dda-de5f-4e3a-a1cd-5d2dbbdc26d8 - - - - -] Updating share status
2020-05-19 13:30:30.139 128181 INFO manila.share.manager [req-75f80a61-41c3-410c-8784-3345b48f3bad - - - - -] Updating share status
2020-05-19 13:30:46.084 128182 INFO manila.share.manager [req-5ae35dda-de5f-4e3a-a1cd-5d2dbbdc26d8 - - - - -] Updating share status
2020-05-19 13:30:46.962 128182 INFO manila.share.manager [req-653b52ec-18bd-4078-b515-11767971c09d cdd65fbe8be24b3fba5bf0ff0f14bc7c 8e9f4288a98b46cc881b9 e8bcc1659f5 - - -] Using preexisting share server: '55882b22-a1ca-450a-8b90-8e44876360e0'
2020-05-19 13:30:57.328 128182 INFO manila.share.manager [req-653b52ec-18bd-4078-b515-11767971c09d cdd65fbe8be24b3fba5bf0ff0f14bc7c 8e9f4288a98b46cc881b9 e8bcc1659f5 - - -] Share instance 2e5a991b-820d-4c2a-8d3b-787e5a21fc7a created successfully.
2020-05-19 13:31:30.136 128181 INFO manila.share.manager [req-75f80a61-41c3-410c-8784-3345b48f3bad - - - - -] Updating share status
在已经存在的虚拟机上挂盘，一个share对应一个盘
[root@openstack4k8s k8s(keystone_services)]# openstack volume list
+--------------------------------------+---------------------------------------------------+--------+------+---------------------------------------------------------------+
| ID | Name | Status | Size | Attached to |
+--------------------------------------+---------------------------------------------------+--------+------+---------------------------------------------------------------+
| 18da7c55-14d2-48ea-8eba-58627ac8bdc9 | manila-share-55ca658e-78f2-4f7e-8955-4d095dfc415c | in-use | 10 | Attached to 55882b22-a1ca-450a-8b90-8e44876360e0 on /dev/vdd |
| 232b1f0d-7f9b-487a-b96c-a3145d1f744d | manila-share-2e5a991b-820d-4c2a-8d3b-787e5a21fc7a | in-use | 5 | Attached to 55882b22-a1ca-450a-8b90-8e44876360e0 on /dev/vdc |
| 90bdb569-3119-4fd8-bf6d-04ebbf6ca211 | manila-share-131f5816-59c1-477f-84a2-e1bfa71eb856 | in-use | 1 | Attached to 55882b22-a1ca-450a-8b90-8e44876360e0 on /dev/vdb |
| 4a6f4fac-ebf5-4d26-81d8-a67b160c63bc | manila-share-bb80d66f-2764-465b-a5bf-ff0ca6047375 | in-use | 1 | Attached to 823fa69a-b9cb-4f72-a72b-43e45fccb3ec on /dev/vdb |
+--------------------------------------+---------------------------------------------------+--------+------+---------------------------------------------------------------+
[root@openstack4k8s k8s(keystone_services)]# ssh manila@10.254.0.23
manila@10.254.0.23's password:
Welcome to Ubuntu 18.04.4 LTS (GNU/Linux 4.15.0-96-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
Last login: Tue May 19 17:32:58 2020 from 10.254.0.22
manila@ubuntu:~$ df
Filesystem 1K-blocks Used Available Use% Mounted on
udev 4061600 0 4061600 0% /dev
tmpfs 816760 2016 814744 1% /run
/dev/vda1 1913908 1127720 664224 63% /
tmpfs 4083788 0 4083788 0% /dev/shm
tmpfs 5120 0 5120 0% /run/lock
tmpfs 4083788 0 4083788 0% /sys/fs/cgroup
tmpfs 816756 0 816756 0% /run/user/1000
/dev/vdb 999320 2564 927944 1% /shares/share-131f5816-59c1-477f-84a2-e1bfa71eb856
/dev/vdc 5095040 20472 4796040 1% /shares/share-2e5a991b-820d-4c2a-8d3b-787e5a21fc7a
/dev/vdd 10255636 36888 9678076 1% /shares/share-55ca658e-78f2-4f7e-8955-4d095dfc415c
manila@ubuntu:~$ sudo fdisk -l
Disk /dev/vda: 2 GiB, 2164391936 bytes, 4227328 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x120ed681
Device Boot Start End Sectors Size Id Type
/dev/vda1 * 2048 4225279 4223232 2G 83 Linux
Disk /dev/vdb: 1 GiB, 1073741824 bytes, 2097152 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/vdc: 5 GiB, 5368709120 bytes, 10485760 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/vdd: 10 GiB, 10737418240 bytes, 20971520 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
manila@ubuntu:~$
share server vm和share network对应关系
[root@openstack4k8s k8s(keystone_services)]# manila share-server-list
+--------------------------------------+-----------------------+--------+---------------------+----------------------------------+----------------------------+--------------------------------------+
| Id | Host | Status | Share Network | Project Id | Updated_at | Share Network Subnet Id |
+--------------------------------------+-----------------------+--------+---------------------+----------------------------------+----------------------------+--------------------------------------+
| 55882b22-a1ca-450a-8b90-8e44876360e0 | openstack4k8s@generic | active | demo-share-network2 | 8e9f4288a98b46cc881b9e8bcc1659f5 | 2020-05-19T17:21:54.000000 | 1e686ebc-d6c6-4f28-a751-6c62811a3176 |
| 823fa69a-b9cb-4f72-a72b-43e45fccb3ec | openstack4k8s@generic | active | demo-share-network1 | 8e9f4288a98b46cc881b9e8bcc1659f5 | 2020-05-18T19:14:46.000000 | 7fba0be0-e7b0-4147-beb7-427cff9b8866 |
+--------------------------------------+-----------------------+--------+---------------------+----------------------------------+----------------------------+--------------------------------------+
[root@openstack4k8s work(keystone_demo)]# manila share-network-list
+--------------------------------------+---------------------+
| id | name |
+--------------------------------------+---------------------+
| 7589692a-852e-4cea-9f2e-d002a20ab813 | demo-share-network2 |
| 80ee729c-a454-4760-9928-2601af555b98 | demo-share-network1 |
+--------------------------------------+---------------------+
[root@openstack4k8s work(keystone_demo)]# manila share-network-show 80ee729c-a454-4760-9928-2601af555b98
+-----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+-----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| description | None |
| name | demo-share-network1 |
| share_network_subnets | [{u'availability_zone': None, u'neutron_net_id': u'9ee4e64a-da83-4cfd-aded-f6542bbd3937', u'created_at': u'2020-05-18T19:08:40.000000', u'neutron_subnet_id': u'cf87d2b1-7fb3-4f90-996a-c7600ed0f27d', u'updated_at': None, u'mtu': None, u'id': u'7fba0be0-e7b0-4147-beb7-427cff9b8866', u'segmentation_id': None, u'ip_version': None, u'cidr': None, u'network_type': None, u'gateway': None}] |
| created_at | 2020-05-18T19:08:40.000000 |
| project_id | 8e9f4288a98b46cc881b9e8bcc1659f5 |
| updated_at | None |
| id | 80ee729c-a454-4760-9928-2601af555b98 |
+-----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[root@openstack4k8s work(keystone_demo)]# manila share-network-subnet-show 80ee729c-a454-4760-9928-2601af555b98 7fba0be0-e7b0-4147-beb7-427cff9b8866
+--------------------+--------------------------------------+
| Property | Value |
+--------------------+--------------------------------------+
| availability_zone | None |
| neutron_net_id | 9ee4e64a-da83-4cfd-aded-f6542bbd3937 |
| created_at | 2020-05-18T19:08:40.000000 |
| neutron_subnet_id | cf87d2b1-7fb3-4f90-996a-c7600ed0f27d |
| updated_at | None |
| share_network_id | 80ee729c-a454-4760-9928-2601af555b98 |
| mtu | None |
| id | 7fba0be0-e7b0-4147-beb7-427cff9b8866 |
| share_network_name | demo-share-network1 |
| segmentation_id | None |
| ip_version | None |
| cidr | None |
| network_type | None |
| gateway | None |
+--------------------+--------------------------------------+
services租户视角
[root@openstack4k8s k8s(keystone_services)]# openstack subnet list
+--------------------------------------+----------------------------------------------------------------------------------------------------+--------------------------------------+------------------+
| ID | Name | Network | Subnet |
+--------------------------------------+----------------------------------------------------------------------------------------------------+--------------------------------------+------------------+
| 71899bf2-c792-48e3-ba7b-a556110e372a | service_subnet_for_handling_of_share_server_for_tenant_subnet_bd9c5c74-a4ac-4859-a480-aadbd213bb7e | 04cb080f-3290-46c3-82a2-3b24bdcfbab0 | 10.254.0.16/28 |
| 778fa9ea-2ef6-463b-894a-f456e90399aa | ext-subnet | 3c83fdd8-c1d6-441b-8163-84d5daa18931 | 192.168.100.0/24 |
| bd9c5c74-a4ac-4859-a480-aadbd213bb7e | subnet_app | a9d09889-e9f5-4b6b-aba0-bcf46959300f | 192.168.2.0/24 |
| cf87d2b1-7fb3-4f90-996a-c7600ed0f27d | subnet_api | 9ee4e64a-da83-4cfd-aded-f6542bbd3937 | 192.168.1.0/24 |
| ff38680c-6a3d-4be9-8e86-cf29b0082942 | service_subnet_for_handling_of_share_server_for_tenant_subnet_cf87d2b1-7fb3-4f90-996a-c7600ed0f27d | 04cb080f-3290-46c3-82a2-3b24bdcfbab0 | 10.254.0.0/28 |
+--------------------------------------+----------------------------------------------------------------------------------------------------+--------------------------------------+------------------+
[root@openstack4k8s k8s(keystone_services)]# openstack network list
+--------------------------------------+------------------------+----------------------------------------------------------------------------+
| ID | Name | Subnets |
+--------------------------------------+------------------------+----------------------------------------------------------------------------+
| 04cb080f-3290-46c3-82a2-3b24bdcfbab0 | manila_service_network | 71899bf2-c792-48e3-ba7b-a556110e372a, ff38680c-6a3d-4be9-8e86-cf29b0082942 |
| 3c83fdd8-c1d6-441b-8163-84d5daa18931 | ext-net | 778fa9ea-2ef6-463b-894a-f456e90399aa |
| 9ee4e64a-da83-4cfd-aded-f6542bbd3937 | net_api | cf87d2b1-7fb3-4f90-996a-c7600ed0f27d |
| a9d09889-e9f5-4b6b-aba0-bcf46959300f | net_app | bd9c5c74-a4ac-4859-a480-aadbd213bb7e |
+--------------------------------------+------------------------+----------------------------------------------------------------------------+
[root@openstack4k8s k8s(keystone_services)]# openstack server list
+--------------------------------------+--------------------------------------+--------+------------------------------------+----------------------+--------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+--------------------------------------+--------+------------------------------------+----------------------+--------+
| ccca9196-d126-4416-9c79-50e6f32806db | 55882b22-a1ca-450a-8b90-8e44876360e0 | ACTIVE | manila_service_network=10.254.0.23 | manila-service-image | k8s |
| 1e4a7b73-df15-4c03-888f-df27df4cd435 | 823fa69a-b9cb-4f72-a72b-43e45fccb3ec | ACTIVE | manila_service_network=10.254.0.8 | manila-service-image | k8s |
+--------------------------------------+--------------------------------------+--------+------------------------------------+----------------------+--------+
2.7 网络隔离问题
不在share network中的租户也能访问该share network的share server
[root@k8s-m2 ~]# ping 10.254.0.8
PING 10.254.0.8 (10.254.0.8) 56(84) bytes of data.
64 bytes from 10.254.0.8: icmp_seq=1 ttl=63 time=1.03 ms
64 bytes from 10.254.0.8: icmp_seq=2 ttl=63 time=0.787 ms
--- 10.254.0.8 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.787/0.909/1.031/0.122 ms
[root@k8s-m2 ~]# ping 10.254.0.23
PING 10.254.0.23 (10.254.0.23) 56(84) bytes of data.
64 bytes from 10.254.0.23: icmp_seq=1 ttl=63 time=1.39 ms
64 bytes from 10.254.0.23: icmp_seq=2 ttl=63 time=0.646 ms
64 bytes from 10.254.0.23: icmp_seq=3 ttl=63 time=0.270 ms
--- 10.254.0.23 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 0.270/0.769/1.393/0.467 ms
因为所有租户子网和manila_service_network子网都挂在一个router上，所以导致不同的租户之间通了
所以需要保证租户网络挂在不同的router上，manila创建共享网络时，会选择租户子网所在的router挂载manila_service_network新建的子网
以在net-app1租户网络上创建共享网络demo-share-network3为例
[root@openstack4k8s work(keystone_demo)]# openstack network list
+--------------------------------------+----------+--------------------------------------+
| ID | Name | Subnets |
+--------------------------------------+----------+--------------------------------------+
| 3c83fdd8-c1d6-441b-8163-84d5daa18931 | ext-net | 778fa9ea-2ef6-463b-894a-f456e90399aa |
| 9ee4e64a-da83-4cfd-aded-f6542bbd3937 | net_api | cf87d2b1-7fb3-4f90-996a-c7600ed0f27d |
| a9d09889-e9f5-4b6b-aba0-bcf46959300f | net_app | bd9c5c74-a4ac-4859-a480-aadbd213bb7e |
| bb628bdb-1964-4b39-aaa7-5f63d0692995 | net-app1 | cdf62ee6-4535-4e3c-8f6b-86a3b82ebd5a |
+--------------------------------------+----------+--------------------------------------+
[root@openstack4k8s work(keystone_demo)]# openstack subnet show cdf62ee6-4535-4e3c-8f6b-86a3b82ebd5a
+-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| allocation_pools | 192.168.3.2-192.168.3.254 |
| cidr | 192.168.3.0/24 |
| created_at | 2020-05-28T10:39:08Z |
| description | |
| dns_nameservers | |
| enable_dhcp | True |
| gateway_ip | 192.168.3.1 |
| host_routes | |
| id | cdf62ee6-4535-4e3c-8f6b-86a3b82ebd5a |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| location | cloud='', project.domain_id=, project.domain_name='Default', project.id='8e9f4288a98b46cc881b9e8bcc1659f5', project.name='demo', region_name='RegionOne', zone= |
| name | subnet_app1 |
| network_id | bb628bdb-1964-4b39-aaa7-5f63d0692995 |
| prefix_length | None |
| project_id | 8e9f4288a98b46cc881b9e8bcc1659f5 |
| revision_number | 0 |
| segment_id | None |
| service_types | |
| subnetpool_id | None |
| tags | |
| updated_at | 2020-05-28T10:39:08Z |
+-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+
[root@openstack4k8s work(keystone_demo)]# manila share-network-list
+--------------------------------------+---------------------+
| id | name |
+--------------------------------------+---------------------+
| 7589692a-852e-4cea-9f2e-d002a20ab813 | demo-share-network2 |
| 80ee729c-a454-4760-9928-2601af555b98 | demo-share-network1 |
| b3b09282-74fb-4c0a-9cbf-65d0b2cff0ad | demo-share-network3 |
+--------------------------------------+---------------------+
[root@openstack4k8s work(keystone_demo)]# manila share-network-show b3b09282-74fb-4c0a-9cbf-65d0b2cff0ad
+-----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+-----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| description | None |
| name | demo-share-network3 |
| share_network_subnets | [{u'availability_zone': None, u'neutron_net_id': u'bb628bdb-1964-4b39-aaa7-5f63d0692995', u'created_at': u'2020-05-28T10:42:54.000000', u'neutron_subnet_id': u'cdf62ee6-4535-4e3c-8f6b-86a3b82ebd5a', u'updated_at': u'2020-05-28T10:43:37.000000', u'mtu': None, u'id': u'e52d6cfa-cf92-452f-9299-a686da11d89d', u'segmentation_id': None, u'ip_version': None, u'cidr': None, u'network_type': None, u'gateway': None}] |
| created_at | 2020-05-28T10:42:54.000000 |
| project_id | 8e9f4288a98b46cc881b9e8bcc1659f5 |
| updated_at | 2020-05-28T10:44:45.000000 |
| id | b3b09282-74fb-4c0a-9cbf-65d0b2cff0ad |
+-----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[root@openstack4k8s work(keystone_demo)]# openstack port list
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------+--------+
| ID | Name | MAC Address | Fixed IP Addresses | Status |
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------+--------+
| 3e867a13-6aeb-492d-a25e-4914831062f5 | | fa:16:3e:4d:9b:07 | ip_address='192.168.2.1', subnet_id='bd9c5c74-a4ac-4859-a480-aadbd213bb7e' | ACTIVE |
| 6141ed86-b83f-4e4e-a56f-1dac2008de09 | | fa:16:3e:4e:35:de | ip_address='192.168.2.2', subnet_id='bd9c5c74-a4ac-4859-a480-aadbd213bb7e' | DOWN |
| 9df2dcde-c81e-4aef-8be3-ded290fe387b | | fa:16:3e:f4:d5:67 | ip_address='192.168.3.2', subnet_id='cdf62ee6-4535-4e3c-8f6b-86a3b82ebd5a' | DOWN |
| c7ddf17d-d7f7-4f5a-b6e2-ff9d3811bbde | | fa:16:3e:59:cc:8b | ip_address='192.168.3.1', subnet_id='cdf62ee6-4535-4e3c-8f6b-86a3b82ebd5a' | ACTIVE |
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------+--------+
共享网络包含的net-app1子网subnet_app1和manila_service_network 子网 都挂在router1上
[root@openstack4k8s sealos(keystone_admin)]# openstack router list
+--------------------------------------+---------+--------+-------+----------------------------------+
| ID | Name | Status | State | Project |
+--------------------------------------+---------+--------+-------+----------------------------------+
| 5c597b98-3510-42b8-8d3e-a5245f2c0863 | router1 | ACTIVE | UP | 23a40789342d4de6bc06d0b0bef7069c |
| cc5fb732-045d-4a9f-9d46-d156a23a7d50 | router | ACTIVE | UP | 23a40789342d4de6bc06d0b0bef7069c |
+--------------------------------------+---------+--------+-------+----------------------------------+
[root@openstack4k8s sealos(keystone_admin)]# openstack router show 5c597b98-3510-42b8-8d3e-a5245f2c0863
+-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | None |
| availability_zones | None |
| created_at | 2020-05-28T10:10:54Z |
| description | |
| external_gateway_info | null |
| flavor_id | None |
| id | 5c597b98-3510-42b8-8d3e-a5245f2c0863 |
| interfaces_info | [{"subnet_id": "cdf62ee6-4535-4e3c-8f6b-86a3b82ebd5a", "ip_address": "192.168.3.1", "port_id": "c7ddf17d-d7f7-4f5a-b6e2-ff9d3811bbde"}, {"subnet_id": "edf4630c-a3be-4ac3-8d9d-8f0a5b68ce50", "ip_address": "10.254.0.33", "port_id": "d9c5a1d7-4b48-493d-b6ed-63de54148e84"}] |
| location | cloud='', project.domain_id=, project.domain_name='Default', project.id='23a40789342d4de6bc06d0b0bef7069c', project.name='admin', region_name='RegionOne', zone= |
| name | router1 |
| project_id | 23a40789342d4de6bc06d0b0bef7069c |
| revision_number | 2 |
| routes | |
| status | ACTIVE |
| tags | |
| updated_at | 2020-05-28T10:47:08Z |
+-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[root@openstack4k8s sealos(keystone_admin)]#
[root@openstack4k8s sealos(keystone_admin)]# openstack network list
+--------------------------------------+------------------------+------------------------------------------------------------------------------------------------------------------+
| ID | Name | Subnets |
+--------------------------------------+------------------------+------------------------------------------------------------------------------------------------------------------+
| 04cb080f-3290-46c3-82a2-3b24bdcfbab0 | manila_service_network | 71899bf2-c792-48e3-ba7b-a556110e372a, edf4630c-a3be-4ac3-8d9d-8f0a5b68ce50, ff38680c-6a3d-4be9-8e86-cf29b0082942 |
| 3c83fdd8-c1d6-441b-8163-84d5daa18931 | ext-net | 778fa9ea-2ef6-463b-894a-f456e90399aa |
| 9ee4e64a-da83-4cfd-aded-f6542bbd3937 | net_api | cf87d2b1-7fb3-4f90-996a-c7600ed0f27d |
| a9d09889-e9f5-4b6b-aba0-bcf46959300f | net_app | bd9c5c74-a4ac-4859-a480-aadbd213bb7e |
| bb628bdb-1964-4b39-aaa7-5f63d0692995 | net-app1 | cdf62ee6-4535-4e3c-8f6b-86a3b82ebd5a |
+--------------------------------------+------------------------+------------------------------------------------------------------------------------------------------------------+
[root@openstack4k8s sealos(keystone_admin)]# openstack subnet list
+--------------------------------------+----------------------------------------------------------------------------------------------------+--------------------------------------+------------------+
| ID | Name | Network | Subnet |
+--------------------------------------+----------------------------------------------------------------------------------------------------+--------------------------------------+------------------+
| 71899bf2-c792-48e3-ba7b-a556110e372a | service_subnet_for_handling_of_share_server_for_tenant_subnet_bd9c5c74-a4ac-4859-a480-aadbd213bb7e | 04cb080f-3290-46c3-82a2-3b24bdcfbab0 | 10.254.0.16/28 |
| 778fa9ea-2ef6-463b-894a-f456e90399aa | ext-subnet | 3c83fdd8-c1d6-441b-8163-84d5daa18931 | 192.168.100.0/24 |
| bd9c5c74-a4ac-4859-a480-aadbd213bb7e | subnet_app | a9d09889-e9f5-4b6b-aba0-bcf46959300f | 192.168.2.0/24 |
| cdf62ee6-4535-4e3c-8f6b-86a3b82ebd5a | subnet_app1 | bb628bdb-1964-4b39-aaa7-5f63d0692995 | 192.168.3.0/24 |
| cf87d2b1-7fb3-4f90-996a-c7600ed0f27d | subnet_api | 9ee4e64a-da83-4cfd-aded-f6542bbd3937 | 192.168.1.0/24 |
| edf4630c-a3be-4ac3-8d9d-8f0a5b68ce50 | service_subnet_for_handling_of_share_server_for_tenant_subnet_cdf62ee6-4535-4e3c-8f6b-86a3b82ebd5a | 04cb080f-3290-46c3-82a2-3b24bdcfbab0 | 10.254.0.32/28 |
| ff38680c-6a3d-4be9-8e86-cf29b0082942 | service_subnet_for_handling_of_share_server_for_tenant_subnet_cf87d2b1-7fb3-4f90-996a-c7600ed0f27d | 04cb080f-3290-46c3-82a2-3b24bdcfbab0 | 10.254.0.0/28 |
+--------------------------------------+----------------------------------------------------------------------------------------------------+--------------------------------------+------------------+
[root@openstack4k8s sealos(keystone_admin)]# openstack subnet show edf4630c-a3be-4ac3-8d9d-8f0a5b68ce50
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+
| allocation_pools | 10.254.0.34-10.254.0.46 |
| cidr | 10.254.0.32/28 |
| created_at | 2020-05-28T10:47:05Z |
| description | |
| dns_nameservers | |
| enable_dhcp | True |
| gateway_ip | 10.254.0.33 |
| host_routes | |
| id | edf4630c-a3be-4ac3-8d9d-8f0a5b68ce50 |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| location | cloud='', project.domain_id=, project.domain_name=, project.id='99671b33717d4be5a6ee2227c39da017', project.name=, region_name='RegionOne', zone= |
| name | service_subnet_for_handling_of_share_server_for_tenant_subnet_cdf62ee6-4535-4e3c-8f6b-86a3b82ebd5a |
| network_id | 04cb080f-3290-46c3-82a2-3b24bdcfbab0 |
| prefix_length | None |
| project_id | 99671b33717d4be5a6ee2227c39da017 |
| revision_number | 0 |
| segment_id | None |
| service_types | |
| subnetpool_id | None |
| tags | |
| updated_at | 2020-05-28T10:47:05Z |
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+
manila_service_network 和tap口之间是一个port，多个ip地址
[root@openstack4k8s k8s(keystone_services)]# openstack port show 512369b0-c969-4760-bebd-de188b7282e5
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | UP |
| allowed_address_pairs | |
| binding_host_id | openstack4k8s |
| binding_profile | |
| binding_vif_details | port_filter='True' |
| binding_vif_type | ovs |
| binding_vnic_type | normal |
| created_at | 2020-05-18T17:15:17Z |
| data_plane_status | None |
| description | |
| device_id | manila-share |
| device_owner | manila:share |
| dns_assignment | None |
| dns_domain | None |
| dns_name | None |
| extra_dhcp_opts | |
| fixed_ips | ip_address='10.254.0.10', subnet_id='ff38680c-6a3d-4be9-8e86-cf29b0082942' |
| | ip_address='10.254.0.22', subnet_id='71899bf2-c792-48e3-ba7b-a556110e372a' |
| | ip_address='10.254.0.40', subnet_id='edf4630c-a3be-4ac3-8d9d-8f0a5b68ce50' |
| id | 512369b0-c969-4760-bebd-de188b7282e5 |
| location | cloud='', project.domain_id=, project.domain_name='Default', project.id='99671b33717d4be5a6ee2227c39da017', project.name='services', region_name='RegionOne', zone= |
| mac_address | fa:16:3e:9c:17:f7 |
| name | |
| network_id | 04cb080f-3290-46c3-82a2-3b24bdcfbab0 |
| port_security_enabled | False |
| project_id | 99671b33717d4be5a6ee2227c39da017 |
| propagate_uplink_status | None |
| qos_policy_id | None |
| resource_request | None |
| revision_number | 6 |
| security_group_ids | |
| status | ACTIVE |
| tags | |
| trunk_details | None |
| updated_at | 2020-05-28T10:47:10Z |
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
886
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
