用Pipework给Docker多容器配置局域网网络

用Pipework给Docker多容器配置局域网网络

遇到问题：用pipework工具给Docker容器指定局域网IP地址，容器无法与局域网、外网通信。

pipework是由Docker的工程师Jérôme Petazzoni开发的一个Docker网络配置工具。Docker自身的网络功能比较简单，不能满足很多复杂的应用场景。因此，有很多开源项目用来改善Docker的网络功能，如pipework、weave、flannel等。
这里记录了，笔者在参考两本书的过程中， 使用pipework配置Docker多容器时遇到问题，以及最后的解决办法。

操作流程：

1）. 从github上下载pipework：

$ git clone https://github.com/jpetazzo/pipework

2）. 开启一个无网络模式指定（–net=none）的容器

$ sudo docker run -it --rm --net=none --name cookbook ubuntu:14.04 bash
root@15afb0c398c5:/# ip -d link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0 
root@15afb0c398c5:/# ifconfig
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

3）. 在另外一个终端窗口使用pipework完成cookbook容器配置

$ sudo pipework/pipework br0 cookbook 192.168.1.10/24@192.168.1.254
[sudo] password for mingchen: 
[mingchen:~]
$ ip -d link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0 
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 64:51:06:5b:c1:7d brd ff:ff:ff:ff:ff:ff promiscuity 0 
3: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default 
    link/ether 6e:25:8c:28:2e:c2 brd ff:ff:ff:ff:ff:ff promiscuity 1 
    openvswitch 
4: ovs0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default 
    link/ether 52:c1:11:73:4b:49 brd ff:ff:ff:ff:ff:ff promiscuity 1 
    openvswitch 
5: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default 
    link/ether 02:42:ee:2d:8f:51 brd ff:ff:ff:ff:ff:ff promiscuity 0 
    bridge 
6: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default 
    link/ether 7e:09:d4:4a:a7:da brd ff:ff:ff:ff:ff:ff promiscuity 0 
    bridge 
8: veth1pl5035@if7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP mode DEFAULT group default qlen 1000
    link/ether 7e:09:d4:4a:a7:da brd ff:ff:ff:ff:ff:ff promiscuity 1 

4）.在容器终端窗口，检查接口eth1，及路由规则

root@15afb0c398c5:/# ip -d link show eth1
7: eth1@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 32:42:00:17:9d:d7 brd ff:ff:ff:ff:ff:ff promiscuity 0 
    veth 
root@15afb0c398c5:/# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         192.168.1.254   0.0.0.0         UG    0      0        0 eth1
192.168.1.0     *               255.255.255.0   U     0      0        0 eth1
root@15afb0c398c5:/# ifconfig            
eth1      Link encap:Ethernet  HWaddr 32:42:00:17:9d:d7  
          inet addr:192.168.1.10  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: fe80::3042:ff:fe17:9dd7/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:49 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:7980 (7.9 KB)  TX bytes:690 (690.0 B)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

5）列举在主机上的network links

 $ ip -d link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0 
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 64:51:06:5b:c1:7d brd ff:ff:ff:ff:ff:ff promiscuity 0 
3: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default 
    link/ether 6e:25:8c:28:2e:c2 brd ff:ff:ff:ff:ff:ff promiscuity 1 
    openvswitch 
4: ovs0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default 
    link/ether 52:c1:11:73:4b:49 brd ff:ff:ff:ff:ff:ff promiscuity 1 
    openvswitch 
5: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default 
    link/ether 02:42:ee:2d:8f:51 brd ff:ff:ff:ff:ff:ff promiscuity 0 
    bridge 
6: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default 
    link/ether 7e:09:d4:4a:a7:da brd ff:ff:ff:ff:ff:ff promiscuity 0 
    bridge 
8: veth1pl5035@if7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP mode DEFAULT group default qlen 1000
    link/ether 7e:09:d4:4a:a7:da brd ff:ff:ff:ff:ff:ff promiscuity 1 
    veth 

6）在主机上添加 NAT masquerading 规则：

[mingchen:~]
$ sudo iptables -t nat -A POSTROUTING -s 192.168.0.0/16 -j MASQUERADE

7）在容器内ping外网和局域网均不通

root@15afb0c398c5:/# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
From 192.168.1.10 icmp_seq=1 Destination Host Unreachable
From 192.168.1.10 icmp_seq=2 Destination Host Unreachable
From 192.168.1.10 icmp_seq=3 Destination Host Unreachable
^Z
[1]+  Stopped                 ping 8.8.8.8
root@15afb0c398c5:/# ping 192.168.1.100
PING 192.168.1.100 (192.168.1.100) 56(84) bytes of data.
From 192.168.1.10 icmp_seq=1 Destination Host Unreachable
From 192.168.1.10 icmp_seq=2 Destination Host Unreachable
From 192.168.1.10 icmp_seq=3 Destination Host Unreachable

问题原因：

忘记把网桥br0配置ip/mask了，也就是配置容器ip和route的网络信息中的route。如不配置，那么容器内部是不能通过br0网桥通信的。另外docker 1.9 以后就支持 overlay网络了。直接用http://www.jianshu.com/p/3eb7448adea0 ， 就可跨主机通信。

解决方式：

需要解决：网络不通问题。

$ sysctl –w net.ipv4.ip_forward=1
$ sudo docker run -it --rm --net=none --name cookbook ubuntu:14.04 bash
$ sudo pipework/pipework br0 cookbook 192.168.1.10/24@192.168.1.1
$ sudo apt-get install bridge-utils
$ sudo brctl  addif br0  eth0
$ ip addr add 192.168.1.1/24 dev br0