Docker 中的网络
1. Docke 中的网络分类
- 单机网络
- Bridge Network
- Host Network
- None Network
- 多机网络
- Bridge Network
2. Docke 网络的基础 NET Namespace
2.1. Container 的 NET Namespace 简介
首先我们启动一个 container
$ docker run -d --name busybox1 busybox /bin/sh -c "while true; do sleep 3600; done"
c7d5d7a23be4f66a807cf3f898ef65308721337c368f452512e4989224128909
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c7d5d7a23be4 busybox "/bin/sh -c 'while..." 10 seconds ago Up 9 seconds busybox
我们查看 container 网络情况
$ docker exec busybox1 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
6: eth0@if7: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue
link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.2/16 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::42:acff:fe11:2/64 scope link
valid_lft forever preferred_lft forever
可以看到这个 container 有两个网络接口,这些便是这个 container 的网络命名空间
我们再次启动一个 container 并查看其网络 namespace
$ docker run -d --name busybox2 busybox /bin/sh -c "while true; do sleep 3600; done"
e5f96d268f93b47cef7984210e06d536c54a264bb08f687ddfcdfe9b436c1115
$ docker exec e5f ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
8: eth0@if9: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue
link/ether 02:42:ac:11:00:03 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.3/16 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::42:acff:fe11:3/64 scope link
valid_lft forever preferred_lft forever
这两个 container 的 net namespace 是各自独立的一套 namespace,并且这两个独立的 net namespace 之间网络互通
$ docker exec busybox1 /bin/sh -c "ping 172.17.0.3"
PING 172.17.0.3 (172.17.0.3): 56 data bytes
64 bytes from 172.17.0.3: seq=0 ttl=64 time=0.077 ms
64 bytes from 172.17.0.3: seq=1 ttl=64 time=0.088 ms
^C
2.2. NET Namespace 实验
- 创建两个 net namespace,并查看是否创建成功
$ ip netns add ns1
$ ip netns add ns2
$ ip netns list
ns2
ns1
- 查看 ns1 的 net namespace 的信息
可以看到 ns1 只有一个本地的回环端口,并且是 DOWN 的状态
$ ip netns exec ns1 ip a
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
我们可以先将这个端口启动起来
$ ip netns exec ns1 ip link set dev lo up
$ ip netns exec ns1 ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
现在这个端口就变成 UNKNOWN 状态
至于为什么会是 UNKNOWN 状态,是由于端口 UP 起来需要满足一个条件就是两端要连起来。就是说单个端口没法 UP 起来的,必须是一对。
- 添加一对 veth-pair
$ ip link add tap1 type veth peer name tap2
$ ip link
... ...
10: tap2@tap1: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
link/ether 0e:fb:bf:12:6a:96 brd ff:ff:ff:ff:ff:ff
11: tap1@tap2: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
link/ether 5e:79:1a:db:5b:f6 brd ff:ff:ff:ff:ff:ff
... ...
- 将这对 veth-pair 添加 到 ns1 和 ns2 这两个 namespace 上
$ ip link set tap1 netns ns1
$ ip link set tap2 netns ns2
我们分别查看这两个 namespace,发现已经绑定成功
$ ip netns exec ns1 ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
11: tap1@if10: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
link/ether 5e:79:1a:db:5b:f6 brd ff:ff:ff:ff:ff:ff link-netnsid 1
$ ip netns exec ns2 ip a
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
10: tap2@if11: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 0e:fb:bf:12:6a:96 brd ff:ff:ff:ff:ff:ff link-netnsid 0
- 接着我们绑定 ip,并分别启动 veth 端口
$ ip netns exec ns1 ip addr add 192.168.1.1/24 dev tap1
$ ip netns exec ns2 ip addr add 192.168.1.2/24 dev tap2
$ ip netns exec ns1 ip link set dev tap1 up
$ ip netns exec ns2 ip link set dev tap2 up
这时可以看到 ip 绑定并且端口启动成功
$ ip netns exec ns1 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
11: tap1@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
link/ether 5e:79:1a:db:5b:f6 brd ff:ff:ff:ff:ff:ff link-netnsid 1
inet 192.168.1.1/24 scope global tap1
valid_lft forever preferred_lft forever
inet6 fe80::5c79:1aff:fedb:5bf6/64 scope link
valid_lft forever preferred_lft forever
- 测试两个 namespace 互通性
$ ip netns exec ns1 ping 192.168.1.2
PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data.
64 bytes from 192.168.1.2: icmp_seq=1 ttl=64 time=0.060 ms
64 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=0.056 ms
^C
$ ip netns exec ns2 ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.041 ms
64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=0.034 ms
^C
这两个 namespace 和上面的两个 busybox 网络互通的原理是一样,以上测试成功。
3. Docker 中的 Bridge Network
首先我们将 busybox2 停掉并且删掉
$ docker rm -f busybox2
docker rm -f busybox2
我们先查看本地 Docker 有哪些网络
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
bda0e0f1ccad bridge bridge local
3852a57cbd5a host host local
a7c283f057ce none null local
再看下 bridge 的详情
$ docker network inspect bda0e0f1ccad
[
{
"Name": "bridge",
"Id": "bda0e0f1ccad1c76f34dfb1d9502abc6e937c324acd65f1b67f8ab7f8c24ba14",
... ...
"Containers": {
"c7d5d7a23be4f66a807cf3f898ef65308721337c368f452512e4989224128909": {
"Name": "busybox1",
"EndpointID": "a4a72e1c6ca50eddc1d9dce181d141ed8307f95daff14b96042ab3c9930925be",
"MacAddress": "02:42:ac:11:00:02",
"IPv4Address": "172.17.0.2/16",
"IPv6Address": ""
}
},
... ...
}
]
其中有一块是 Containers ,包括了一个个 container,并且 ip 是 172.17.0.2
也就是说明我们第一步创建的 busybox1 连接到了这个 bridge 网络上面。
然后我们再看下本机的网络
$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:16:3e:0a:c1:1f brd ff:ff:ff:ff:ff:ff
inet 172.16.115.198/20 brd 172.16.127.255 scope global dynamic eth0
valid_lft 314668192sec preferred_lft 314668192sec
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 02:42:05:80:9b:c6 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 scope global docker0
valid_lft forever preferred_lft forever
7: veth8c6f53e@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP
link/ether 4a:8a:33:15:a2:96 brd ff:ff:ff:ff:ff:ff link-netnsid 0
这里的 docker0 网络是本地的 namespace。还有一个veth接口。
我们再看下 busybox 的网络情况
$ docker exec busybox ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
6: eth0@if7: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue
link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.2/16 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::42:acff:fe11:2/64 scope link
valid_lft forever preferred_lft forever
其实,这个 container 里的 eth0 和 本机的 veth8c6f53e 是一对。这个 container 通过这一对 veth 连接到本机的网络上面,也就是那个 docker0 上。
下面,我们来验证一下
先安装个小工具
$ yum install -y bridge-utils
查看 namespace
$ brctl show
bridge name bridge id STP enabled interfaces
docker0 8000.024205809bc6 no veth8c6f53e
可以看出 docker0 这个 namespace 只有一个端口是 veth8c6f53e 正好验证了上面的说法。
现在我们再重新创建一个 container,再看下 docker 的网络
$ docker network inspect bridge
[
{
"Name": "bridge",
"Id": "676883a409d087da0ec4bbb2820b3c2f205291aa7da0135b784d73f96559adf8",
"Created": "2019-11-01T13:29:48.00442962+08:00",
"Scope": "local",
"Driver": "bridge",
... ...
"Containers": {
"58217eef521d4e4c2e643ebee9692014c0dcc72d3d58a150bc9ae3d5ea5ca923": {
"Name": "busybox2",
"EndpointID": "9ec0d5f29529c2d6c4febe7accb957cba6e3e120eaaf65de5648cf0b9534c344",
"MacAddress": "02:42:ac:11:00:03",
"IPv4Address": "172.17.0.3/16",
"IPv6Address": ""
},
"c7d5d7a23be4f66a807cf3f898ef65308721337c368f452512e4989224128909": {
"Name": "busybox1",
"EndpointID": "a4a72e1c6ca50eddc1d9dce181d141ed8307f95daff14b96042ab3c9930925be",
"MacAddress": "02:42:ac:11:00:02",
"IPv4Address": "172.17.0.2/16",
"IPv6Address": ""
}
},
... ...
}
]
可以看到多了一个刚刚创建的 busybox2 容器连接到了 bridge 上
我们再次在本机上查看下网络
$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:16:3e:0a:c1:1f brd ff:ff:ff:ff:ff:ff
inet 172.16.115.198/20 brd 172.16.127.255 scope global dynamic eth0
valid_lft 314667111sec preferred_lft 314667111sec
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 02:42:05:80:9b:c6 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 scope global docker0
valid_lft forever preferred_lft forever
7: veth8c6f53e@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP
link/ether 4a:8a:33:15:a2:96 brd ff:ff:ff:ff:ff:ff link-netnsid 0
13: veth835d4ff@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP
link/ether a6:b4:d7:28:15:c0 brd ff:ff:ff:ff:ff:ff link-netnsid 1
这时又多了一个端口 veth835d4ff,我们再看下 namespace
$ brctl show
bridge name bridge id STP enabled interfaces
docker0 8000.024205809bc6 no veth835d4ff
veth8c6f53e
参考资料