docker overlay network测试

最新推荐文章于 2024-08-02 07:59:27 发布

ARunningPig

最新推荐文章于 2024-08-02 07:59:27 发布

阅读量8k

点赞数 1

分类专栏： docker 文章标签： centos 网络 overlay docker vxlan

本文链接：https://blog.csdn.net/asd05txffh/article/details/51130197

版权

docker 专栏收录该内容

9 篇文章 0 订阅

订阅专栏

docker自1.9版本后，引入了overlay网络（本文不具体分析其背后使用的技术）。重点解决之前docker网络在跨主机通信方面的不足。本文记录，参考官方指导文档，搭建测试overlay网络的过程。
文中使用的os为centos7，内核版本为3.10。而docker 1.9版本overlay网络要求内核版本在3.19以上。自docker 1.10版本后，docker overlay 支持3.10版本内核。因此，本文中使用的docker版本为1.10.3。
测试环境工包含三台virtualbox虚拟机。1台作为key-value store存储的机器，本文测试中选用的是etcd。另外，两台用来测试跨host的通信,分别为net1和net2。
docker info：

[root@net1 vagrant]# docker  info
Containers: 1
 Running: 1
 Paused: 0
 Stopped: 0
Images: 1
Server Version: 1.10.3
 Storage Driver: devicemapper
 Pool Name: docker-253:0-469034-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 10.74 GB
 Backing Filesystem: xfs
 Data file: /dev/vg-docker/data
 Metadata file: /dev/vg-docker/metadata
 Data Space Used: 41.16 MB
 Data Space Total: 10.74 GB
 Data Space Available: 10.7 GB
 Metadata Space Used: 761.9 kB
 Metadata Space Total: 10.63 GB
 Metadata Space Available: 10.63 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Library Version: 1.02.107-RHEL7 (2015-12-01)
Execution Driver: native-0.2
Logging Driver: json-file
Plugins: 
 Volume: local
 Network: null host overlay bridge
Kernel Version: 3.10.0-229.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 993.6 MiB
Name: net1
ID: TU6M:E6WM:PZDN:ULJX:EWKS:UPLQ:Z54D:XP52:64C7:Z4XN:TJ76:VG7O
WARNING: bridge-nf-call-ip6tables is disabled
Cluster store: etcd://172.28.0.2:4001
Cluster advertise: 172.28.0.3:0

注：使用默认的loop的时候，overlay网络测试存在问题。无法创建成功。
参考官方文档，需要配置docker daemon的如下参数：

--cluster-store=PROVIDER://URL Describes the location of the KV service.
--cluster-advertise=HOST_IP|HOST_IFACE:PORT
The IP address or interface of the HOST used for clustering.
--cluster-store-opt=KEY-VALUE OPTIONS
Options such as TLS certificate or tuning discovery Timers

docker daemon参数

/usr/bin/docker daemon -H fd:// --storage-driver=devicemapper --storage-opt dm.datadev=/dev/vg-docker/data --storage-opt dm.metadatadev=/dev/vg-docker/metadata  --cluster-store=etcd://172.28.0.2:4001 --cluster-advertise=eth1:0

端口，7946控制面，4789数据面。

firewall-cmd --permanent --add-port=7946/tcp
firewall-cmd --permanent --add-port=7946/udp
firewall-cmd --permanent --add-port=4789/udp

配置完上述参数后，即可创建网络。

docker network create -d overlay mutihost

[root@net1 vagrant]# docker  network ls
NETWORK ID          NAME                DRIVER
15bb57daf277        multihost           overlay             
3cd7ab7018e9        docker_gwbridge     bridge              
a874aa0d9e0b        bridge              bridge              
9fe04ff37f6f        none                null                
010a53c2bf04        host                host 

[root@net1 vagrant]# docker  network inspect multihost
[
    {
        "Name": "multihost",
        "Id": "15bb57daf27731da102c8a5c5bf903e574daa33f5286e938009734a8cd5ce93c",
        "Scope": "global",
        "Driver": "overlay",
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "10.0.0.0/24",
                    "Gateway": "10.0.0.1/24"
                }
            ]
        },
        "Containers": {},
        "Options": {}
    }
]

[root@net2 vagrant]# docker network inspect multihost
[
    {
        "Name": "multihost",
        "Id": "15bb57daf27731da102c8a5c5bf903e574daa33f5286e938009734a8cd5ce93c",
        "Scope": "global",
        "Driver": "overlay",
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "10.0.0.0/24",
                    "Gateway": "10.0.0.1/24"
                }
            ]
        },
        "Containers": {
            "37162168dca4ad715d12f6bc78d1bf0678ff9128fe5d55178e39ed08e847f80a": {
                "Name": "tender_kirch",
                "EndpointID": "765451d1201d570c626470d16c92515de55f4ea1df2a58a03bef8e1767873897",
                "MacAddress": "02:42:0a:00:00:05",
                "IPv4Address": "10.0.0.5/24",
                "IPv6Address": ""
            }
        },
        "Options": {}
    }
]


[root@net2 vagrant]# docker  run -it --rm=true --net=multihost centos /bin/bash
[root@37162168dca4 /]# ping 10.0.0.3
PING 10.0.0.3 (10.0.0.3) 56(84) bytes of data.
64 bytes from 10.0.0.3: icmp_seq=1 ttl=64 time=0.505 ms
64 bytes from 10.0.0.3: icmp_seq=2 ttl=64 time=0.619 ms
64 bytes from 10.0.0.3: icmp_seq=3 ttl=64 time=0.632 ms
64 bytes from 10.0.0.3: icmp_seq=4 ttl=64 time=0.660 ms
64 bytes from 10.0.0.3: icmp_seq=5 ttl=64 time=0.663 ms

观察容器内的链路

[root@37162168dca4 /]# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
11: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT 
    link/ether 02:42:0a:00:00:05 brd ff:ff:ff:ff:ff:ff
13: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT 
    link/ether 02:42:ac:12:00:02 brd ff:ff:ff:ff:ff:ff
[root@37162168dca4 /]# ethtool -S eth1
NIC statistics:
     peer_ifindex: 14
[root@37162168dca4 /]# ethtool -S eth0
NIC statistics:
     peer_ifindex: 12

host上的网桥

[root@net2 vagrant]# brctl show
bridge name bridge id       STP enabled interfaces
docker0     8000.024297afd372   no      
docker_gwbridge     8000.0242117ceeda   no      veth2cef6db
ov-000100-15bb5     8000.96f96b0c7379   no      vetha6b50db
                            vx-000100-15bb5
[root@net2 vagrant]# ip -d link
12: vetha6b50db: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master ov-000100-15bb5 state UP mode DEFAULT 
    link/ether be:c0:b4:39:e5:fc brd ff:ff:ff:ff:ff:ff promiscuity 1 
    veth addrgenmode eui64 
14: veth2cef6db: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker_gwbridge state UP mode DEFAULT 
    link/ether ca:d6:b0:d9:e9:5c brd ff:ff:ff:ff:ff:ff promiscuity 1 
    veth addrgenmode eui64 
                            vx-000100-15bb5
[root@net2 vagrant]# ip -d link show vx-000100-15bb5
10: vx-000100-15bb5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ov-000100-15bb5 state UNKNOWN mode DEFAULT 
    link/ether 96:f9:6b:0c:73:79 brd ff:ff:ff:ff:ff:ff promiscuity 1 
    vxlan id 256 srcport 0 0 dstport 4789 proxy l2miss l3miss ageing 300 addrgenmode eui64

从容器内查询的结果看，链接overaly网络的时候，会创建两个网桥，其中ov-000100-15bb5 网桥用来中有两个设备,一个veth peer用来链接容器和网桥，还有一个vxlan设备，从查询数据看vxlan id为256。
另外，还有一个docker_gwbridge网桥，容器也通过veth pair设备连接到了该网桥。这个网络的作用主要是方便容器对外提供服务。