kubernetes------error：flanneld开启后发现容器间不能互通，包全丢

环境

ifconfig 
docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.47.1  netmask 255.255.255.0  broadcast 172.17.47.255
    
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.238.82  netmask 255.255.255.0  broadcast 192.168.238.255
      
flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 172.17.47.0  netmask 255.255.255.255  broadcast 0.0.0.0
     
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        
virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255

ifconfig
docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.62.1  netmask 255.255.255.0  broadcast 172.17.62.255

ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.238.83  netmask 255.255.255.0  broadcast 192.168.238.255

flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 172.17.62.0  netmask 255.255.255.255  broadcast 0.0.0.0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255

问题

容器间docker0不能互通，包全丢了

ping 172.17.62.1
PING 172.17.62.1 (172.17.62.1) 56(84) bytes of data.
^C
--- 172.17.62.1 ping statistics ---
19 packets transmitted, 0 received, 100% packet loss, time 18008ms

原因

route -n 查看没有问题，最后发现是防火墙没关

node1

route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.238.2   0.0.0.0         UG    100    0        0 ens33
172.17.47.0     0.0.0.0         255.255.255.0   U     0      0        0 docker0
172.17.62.0     172.17.62.0     255.255.255.0   UG    0      0        0 flannel.1
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0

node02

route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.238.2   0.0.0.0         UG    100    0        0 ens33
172.17.47.0     172.17.47.0     255.255.255.0   UG    0      0        0 flannel.1
172.17.62.0     0.0.0.0         255.255.255.0   U     0      0        0 docker0
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0
192.168.238.0   0.0.0.0         255.255.255.0   U     100    0        0 ens33

解决

systemctl stop firewalld.service 
setenforce 0
setenforce: SELinux is disabled

ping 172.17.47.1
PING 172.17.47.1 (172.17.47.1) 56(84) bytes of data.
64 bytes from 172.17.47.1: icmp_seq=1 ttl=64 time=0.238 ms
64 bytes from 172.17.47.1: icmp_seq=2 ttl=64 time=0.186 ms
^C
--- 172.17.47.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.186/0.212/0.238/0.026 ms