浅谈DockerOverlay网络

最新推荐文章于 2024-10-01 22:54:40 发布

如果悲伤有颜色，那么一定是黄昏

最新推荐文章于 2024-10-01 22:54:40 发布

阅读量42

点赞数

文章标签：网络

本文链接：https://blog.csdn.net/Day_Day_No_Bug/article/details/132979619

版权

theme: mk-cute

这是我参与更文挑战的第10天，活动详情查看：更文挑战

DockerOverlay网络

Overlay是一种覆盖在主机上的虚拟网络，我理解的overlay他需要借助插件或工具完成对数据的保存和转发。我们这里使用etcd来完成overlay的跨主机容器通信

1. 安装etcd(这里我只启动两个节点的etcd,也可单机模式)

我这里选择用安装包方式，你也可以选择yum

``` wget https://github.com/etcd-io/etcd/releases/download/v3.5.0/etcd-v3.5.0-linux-amd64.tar.gz

scp etcd-v3.5.0-linux-amd64.tar.gz root@172.16.3.142:/root ```

参考官网

以下操作两个主机均需要进行

tar -zxvf etcd-v3.5.0-linux-amd64.tar.gz cd etcd-v3.5.0-linux-amd64 cp etcdctl /usr/local/bin/

```

我这里遇到一个坑，一定要加上--enable-v2否则会报错，无法创建overlay网络

nohup ./etcd --name docker-node1 --initial-advertise-peer-urls http://172.16.3.141:2380 --listen-peer-urls http://172.16.3.141:2380 --listen-client-urls http://172.16.3.141:2379,http://127.0.0.1:2379 --advertise-client-urls http://172.16.3.141:2379 --initial-cluster-token etcd-cluster --initial-cluster docker-node1=http://172.16.3.141:2380,docker-node2=http://172.16.3.142:2380 --enable-v2 --initial-cluster-state new 2>&1 &

nohup ./etcd --name docker-node2 --initial-advertise-peer-urls http://172.16.3.142:2380 --listen-peer-urls http://172.16.3.142:2380 --listen-client-urls http://172.16.3.142:2379,http://127.0.0.1:2379 --advertise-client-urls http://172.16.3.142:2379 --initial-cluster-token etcd-cluster --initial-cluster docker-node1=http://172.16.3.141:2380,docker-node2=http://172.16.3.142:2380 --enable-v2 --initial-cluster-state new 2>&1 &

etcdctl member list 查看一下节点信息

```

分别添加配置，指定外部存储使用etcd ``` vi /etc/docker/daemon.json

{ "cluster-store":"etcd://172.16.3.141:2379", "cluster-advertise":"172.16.3.141:2375" } ``` 一定要重启下

systemctl daemon-reload service docker restart

2. 创建overlay network

```

也可指定subnet

docker network create -d overlay etcdnet 可以看到另一台也出现了etcdnet docker network ls NETWORK ID NAME DRIVER SCOPE 810ca8a6e002 bridge bridge local 19234a37ec11 etcdnet overlay global ```

我们来看一下etcd中存储的数据格式,可以看到节点信息和子网的配置都有了 [root@localhost etcd-v3.5.0-linux-amd64]# etcdctl ls /docker [root@localhost etcd-v3.5.0-linux-amd64]# etcdctl ls /docker /docker/nodes /docker/network [root@localhost etcd-v3.5.0-linux-amd64]# etcdctl ls /docker/nodes /docker/nodes/172.16.3.141:2375 /docker/nodes/172.16.3.142:2375 [root@localhost etcd-v3.5.0-linux-amd64]# etcdctl ls /docker/network/v1.0 /docker/network/v1.0/endpoint /docker/network/v1.0/network /docker/network/v1.0/endpoint_count /docker/network/v1.0/ipam /docker/network/v1.0/idm /docker/network/v1.0/overlay [root@localhost etcd-v3.5.0-linux-amd64]# etcdctl get /docker/network/v1.0/overlay/network/19234a37ec11b48a6d1491d99264825ee6452fc3007d0841429ec5ca85c94d64 {"mtu":0,"secure":false,"subnets":[{"SubnetIP":"10.0.9.0/24","GwIP":"10.0.9.1/24","Vni":0}]}

3. 使用overlay network创建容器

在141上启动busybox0 在142上启动busybox1 docker run -it -d --name busybox0 --network etcdnet busybox docker run -it -d --name busybox1 --network etcdnet busybox 查看下两个节点的ip分配情况 docker network inspect etcdnet "Containers": { "67845aca4909ec68518ee09a5de72c12472bdd5ab044c3df180df85012b381e8": { "Name": "busybox0", "EndpointID": "e90dff8605bbe7c7f7db508a6b576d738ddb5faf1e7564191c068134c8c79312", "MacAddress": "02:42:0a:00:09:02", "IPv4Address": "10.0.9.2/24", "IPv6Address": "" }, "ep-fc9be25e3f8d99df596d59e283fd0237834bb6ace2b72f3182181f132fd51bbd": { "Name": "busybox1", "EndpointID": "fc9be25e3f8d99df596d59e283fd0237834bb6ace2b72f3182181f132fd51bbd", "MacAddress": "02:42:0a:00:09:03", "IPv4Address": "10.0.9.3/24", "IPv6Address": "" } }, 我们进入容器看一下网卡信息，发现有两个网卡，这里eth0是overlay的eth1是bridge，从上篇文章的docker0可知默认的bridgeIp不应该是172.17.0.1网段上分配的吗？172.18.0.2又是哪里来的？ ``` [root@localhost etcd-v3.5.0-linux-amd64]# docker exec -it busybox0 sh -c 'ifconfig' eth0 Link encap:Ethernet HWaddr 02:42:0A:00:09:02 inet addr:10.0.9.2 Bcast:10.0.9.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1450 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

eth1 Link encap:Ethernet HWaddr 02:42:AC:12:00:02 inet addr:172.18.0.2 Bcast:172.18.255.255 Mask:255.255.0.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:10 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:836 (836.0 B) TX bytes:0 (0.0 B)

lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) 我们在运行该容器的主机上查看一下network,发现突然多了一个网络配置叫docker_gwbridge是一个bridge类型的 [root@localhost etcd-v3.5.0-linux-amd64]# docker network ls NETWORK ID NAME DRIVER SCOPE cd915be3ae23 bridge bridge local 84d1aff11b90 dockergwbridge bridge local 19234a37ec11 etcdnet overlay global 看一下docker_gwbridge对应的详细信息,和容器内的eth1是可以对应上的 docker network inspect 84d1aff11b90 "Containers": { "67845aca4909ec68518ee09a5de72c12472bdd5ab044c3df180df85012b381e8": { "Name": "gatewaya6711fa937d4", "EndpointID": "4ec1a016078d849ae46012c54a712caa7cc64312ebbd78facaecb15e387bda21", "MacAddress": "02:42:ac:12:00:02", "IPv4Address": "172.18.0.2/16", "IPv6Address": "" } }, ```

4. overlay network测试

在141机器上运行的busbox0去ping142机器上运行的busbox1,可以看到无论是ping容器名还是ip都可以 ``` [root@docker01 etcd-v3.5.0-linux-amd64]# docker exec -it busybox0 sh -c 'ping busybox1' PING busybox1 (10.0.9.3): 56 data bytes 64 bytes from 10.0.9.3: seq=0 ttl=64 time=0.499 ms 64 bytes from 10.0.9.3: seq=1 ttl=64 time=0.902 ms 64 bytes from 10.0.9.3: seq=2 ttl=64 time=1.757 ms

[root@docker01 etcd-v3.5.0-linux-amd64]# docker exec -it busybox0 sh -c 'ping 10.0.9.3' PING 10.0.9.3 (10.0.9.3): 56 data bytes 64 bytes from 10.0.9.3: seq=0 ttl=64 time=0.684 ms 64 bytes from 10.0.9.3: seq=1 ttl=64 time=1.390 ms 反过来试试也是没问题的 [root@docker02 etcd-v3.5.0-linux-amd64]# docker exec -it busybox1 sh -c 'ping busybox0' PING busybox0 (10.0.9.2): 56 data bytes 64 bytes from 10.0.9.2: seq=0 ttl=64 time=0.563 ms 64 bytes from 10.0.9.2: seq=1 ttl=64 time=0.959 ms

[root@docker02 etcd-v3.5.0-linux-amd64]# docker exec -it busybox1 sh -c 'ping 10.0.9.2' PING 10.0.9.2 (10.0.9.2): 56 data bytes 64 bytes from 10.0.9.2: seq=0 ttl=64 time=0.638 ms 至于为什么能解析容器名为ip呢，这也很容易想到，这些信息etcd中都有，创建的时候添加一条规则即可 [root@docker02 etcd-v3.5.0-linux-amd64]# docker exec -it busybox1 sh -c 'cat /etc/hosts' 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters 10.0.9.3 8139c23b0e57 ```

5. overly总结

最后测试阶段发现容器互相ping是ping不通的，我很奇怪，去查了一些资料似乎没什么结果，最后升级了一下内核版本从3.10升级到了5.12.13-1.el7.elrepo.x86_64就可以了

6. 没想到吧总结之后还有内容,哎我就是玩

从上一篇docker网络浅析中我们知道网桥模式的玩法，我们这里尝试不使用任何插件来实现跨主机容器互联

首先我们先改一下两台机器的docker0的ip地址,默认是172.17.0.1,不改的话就导致都路由到一个网段了

vi /etc/docker/daemon.json "bip":"172.17.3.100/24"

vi /etc/docker/daemon.json "bip":"172.17.4.100/24" 两天机器分别都改一下然后刷新重启 //主机141 docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 172.17.3.100 netmask 255.255.255.0 broadcast 172.25.0.255 inet6 fe80::42:5aff:fe07:e6d6 prefixlen 64 scopeid 0x20<link> //主机142 docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 172.17.4.100 netmask 255.255.255.0 broadcast 172.25.0.255 inet6 fe80::42:5aff:fe07:e6d6 prefixlen 64 scopeid 0x20<link> 然后我们添加两条路由规则 ``` //主机141 route add -net 172.17.4.0 netmask 255.255.255.0 gw 172.16.3.142 //主机142 route add -net 172.17.3.0 netmask 255.255.255.0 gw 172.16.3.141

//防止丢包两边分别加上 iptables -I DOCKER --dst 172.17.0.0/16 -j ACCEPT ```

我们尝试从141机器的172.17.3.0网段的容器，请求142机器的172.17.4.0网段的容器，看看能否成功 //141 docker run -it -d --name t1 busybox -> ip = 172.17.3.1 //142 docker run -it -d --name t2 busybox -> ip = 172.17.4.1 ``` [root@docker01 etcd-v3.5.0-linux-amd64]# docker exec -it t1 sh -c 'ping 172.17.4.1' PING 172.17.4.1 (172.17.4.1): 56 data bytes 64 bytes from 172.17.4.1: seq=0 ttl=62 time=2.386 ms 64 bytes from 172.17.4.1: seq=1 ttl=62 time=2.061 ms

[root@docker02 etcd-v3.5.0-linux-amd64]# docker exec -it t2 sh -c 'ping 172.17.3.1' PING 172.17.3.1 (172.17.3.1): 56 data bytes 64 bytes from 172.17.3.1: seq=0 ttl=62 time=1.144 ms 64 bytes from 172.17.3.1: seq=1 ttl=62 time=1.606 ms ```