docker network case with customized bridge

最新推荐文章于 2024-08-20 15:24:59 发布

weixin_34056162

最新推荐文章于 2024-08-20 15:24:59 发布

阅读量71

点赞数

文章标签：运维

原文链接：https://yq.aliyun.com/articles/8991

版权

昨天一为朋友问我能不能让container和本机网络在同一个段, 当然是可以的.

首先docker安装好后, 会创建一个网桥, 默认网桥在172.17..42.1/16网段, 启动 container时, 会自动新建link, 并将link interface添加到桥里面, 分配172.17.0.0/16的IP.

那么要让container和外部网络通信的话, 一般是DNAT, 这样效率会很低下, 见我的一篇BLOG.

另一种比较简单的方法是将本机物理网卡加入虚拟网桥, 这样的话, 也可以打通物理机和container的通讯, (甚至可以使用ovs来管理VLAN).

做法如下 :

当前桥 :

[root@db-172-16-3-221 ~]# brctl show
bridge name     bridge id               STP enabled     interfaces
docker0         8000.000000000000       no

当前网络

[root@db-172-16-3-221 ~]# ifconfig
docker0   Link encap:Ethernet  HWaddr D2:61:28:62:E9:8E  
          inet addr:172.17.42.1  Bcast:0.0.0.0  Mask:255.255.0.0
          inet6 addr: fe80::d061:28ff:fe62:e98e/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:468 (468.0 b)

eth0      Link encap:Ethernet  HWaddr 5C:F3:FC:94:12:00  
          inet addr:172.16.3.221  Bcast:172.16.3.255  Mask:255.255.255.0
          inet6 addr: fe80::5ef3:fcff:fe94:1200/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2495587 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10409745 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:265858486 (253.5 MiB)  TX bytes:15226870694 (14.1 GiB)

当前链路

[root@db-172-16-3-221 ~]# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 5c:f3:fc:94:12:00 brd ff:ff:ff:ff:ff:ff
3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 5c:f3:fc:94:12:02 brd ff:ff:ff:ff:ff:ff
4: usb0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
    link/ether 5e:f3:fc:95:12:03 brd ff:ff:ff:ff:ff:ff
5: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether d2:61:28:62:e9:8e brd ff:ff:ff:ff:ff:ff

当前链路地址

[root@db-172-16-3-221 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 5c:f3:fc:94:12:00 brd ff:ff:ff:ff:ff:ff
    inet 172.16.3.221/24 brd 172.16.3.255 scope global eth0
    inet6 fe80::5ef3:fcff:fe94:1200/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 5c:f3:fc:94:12:02 brd ff:ff:ff:ff:ff:ff
4: usb0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
    link/ether 5e:f3:fc:95:12:03 brd ff:ff:ff:ff:ff:ff
    inet 169.254.95.120/24 brd 169.254.95.255 scope global usb0
    inet6 fe80::5cf3:fcff:fe95:1203/64 scope link 
       valid_lft forever preferred_lft forever
5: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether d2:61:28:62:e9:8e brd ff:ff:ff:ff:ff:ff
    inet 172.17.42.1/16 scope global docker0
    inet6 fe80::d061:28ff:fe62:e98e/64 scope link 
       valid_lft forever preferred_lft forever

使用brctl

[root@db-172-16-3-221 ~]# brctl 
Usage: brctl [commands]
commands:
        addbr           <bridge>                add bridge
        delbr           <bridge>                delete bridge
        addif           <bridge> <device>       add interface to bridge
        delif           <bridge> <device>       delete interface from bridge
        setageing       <bridge> <time>         set ageing time
        setbridgeprio   <bridge> <prio>         set bridge priority
        setfd           <bridge> <time>         set bridge forward delay
        sethello        <bridge> <time>         set hello time
        setmaxage       <bridge> <time>         set max message age
        sethashel       <bridge> <int>          set hash elasticity
        sethashmax      <bridge> <int>          set hash max
        setmclmc        <bridge> <int>          set multicast last member count
        setmcrouter     <bridge> <int>          set multicast router
        setmcsnoop      <bridge> <int>          set multicast snooping
        setmcsqc        <bridge> <int>          set multicast startup query count
        setmclmi        <bridge> <time>         set multicast last member interval
        setmcmi         <bridge> <time>         set multicast membership interval
        setmcqpi        <bridge> <time>         set multicast querier interval
        setmcqi         <bridge> <time>         set multicast query interval
        setmcqri        <bridge> <time>         set multicast query response interval
        setmcqri        <bridge> <time>         set multicast startup query interval
        setpathcost     <bridge> <port> <cost>  set path cost
        setportprio     <bridge> <port> <prio>  set port priority
        setportmcrouter <bridge> <port> <int>   set port multicast router
        show            [ <bridge> ]            show a list of bridges
        showmacs        <bridge>                show a list of mac addrs
        showstp         <bridge>                show bridge stp info
        stp             <bridge> {on|off}       turn stp on/off

实际的话, 建议新建一个网桥, 不要用docker0, docker0会被dockerinit处理, 有一些默认的东西.

注意操作时要添加路由, 接口, 否则网络会中断, 那就得登录到终端去操作了.

# vi network.sh
#!/bin/bash

brctl addbr br0
ip link set dev br0 up
ip addr del 172.16.3.221/24 dev eth0
ip addr add 172.16.3.221/24 dev br0
brctl addif br0 eth0
ip route add default via 172.16.3.1 dev br0
ip link set dev docker0 down
brctl delbr docker0

# . ./network

# vi /etc/resolv.conf
nameserver 202.101.172.35

除此之外, 还需要修改docker服务文件.

# vi /etc/init.d/docker
例如 : 
        $exec -d -g /data01/docker &>> $logfile &
改成
        $exec -d -b="br0" -g /data01/docker &>> $logfile &

改完后, 最好把network服务关掉, 配置文件也重命名, 用脚本来启动网络.

将命令加入/etc/rc.local

# chkconfig network off
# chkconfig NetworkManager off
# chkconfig docker off

# vi /etc/rc.local
brctl addbr br0
ip link set dev eth0 up
ip link set dev br0 up
ip addr add 172.16.3.221/24 dev br0
brctl addif br0 eth0
ip route add default via 172.16.3.1 dev br0
service docker start

[root@db-172-16-3-221 ~]# cd /etc/sysconfig/network-scripts/
[root@db-172-16-3-221 network-scripts]# mv ifcfg-eth0 ifcfg-eth0.bak

重启docker 服务, container 将以172.16.3.0/24网段来分配IP.

接下来问题来了, 因为docker container的网络地址是自动分配的, 如果有多台宿主机的话, 使用以上架构, 非常有可能出现IP冲突的情况(如果docker没有判断的话), 建议宿主机使用不同网段, 但是管理就太麻烦了, 所以docker 默认是DNAT通讯的, 省去了网络管理的繁杂, 如果是openstack的话, container的网络是由openstack集中管理的, 而不是docker自动分配, 所以也不会出现冲突的问题.

那么需要使用netns来对docker container进行网络设置.

参考 http://blog.163.com/digoal@126/blog/static/16387704020141062106481/

比较靠谱的方法 :

在主机上创建多个网桥, 一个由openvswitch管理, (目前docker还未直接支持openvswitch接口, 所以需要手工来管理)

将物理网卡添加到ovs管理的网桥, 并在物理交换机相连的接口使用trunck接口(以接收多个VLAN的包).

另一个网桥由bridge-utils管理, (docker目前仅支持bridge接口)

将物理网卡和bridge link加入ovs管理的网桥. 并分配不同的VLAN号. (隔离container和物理网段).

container和物理网段通信或对外通信交给路由器负责.

docker network case with customized bridge - 德哥@Digoal - PostgreSQL research

[参考]

1. man brctl

2. man ip

weixin_34056162

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
docker network case with customized bridge

昨天一为朋友问我能不能让container和本机网络在同一个段, 当然是可以的.首先docker安装好后, 会创建一个网桥, 默认网桥在172.17..42.1/16网段, 启动 container时, 会自动新建link, 并将link interface添加到桥里面, 分配172.17.0.0/16的IP.那么要让container和外部网络通信...
复制链接

扫一扫