k8s之多节点，多pod，duoservice之间通信实验

一、这个问题困扰了好几天，单个节点的pod之间通信没问题，但是当涉及到多个节点之间的通信就不行。最后问题锁定在flannel打通不同节点之间的网络的问题。

二、实验环境

1、ubuntu-14.04、k8s-1.4、flannel-0.5.5、etcd-2.3、docker-1.12.3

2、k8s集群总共有三个节点，一个master（192.168.110.151，同时作为私有registry仓库），minion1（192.168.110.152），minion2（192.168.110.154）

三、实验过程

1、配置集群相关软件

（1）master的etcd（放在/home/docker/xu/etcd目录下）

root@master:/home/docker/xu/etcd# tree
.
├── etcd
├── etcd0.etcd
│   └── member
│       ├── snap
│       │   ├── 000000000000005c-0000000000013889.snap
│       │   ├── 0000000000000085-0000000000015f9a.snap
│       │   ├── 00000000000000bc-00000000000186ab.snap
│       │   ├── 00000000000000bf-000000000001adbc.snap
│       │   └── 00000000000000bf-000000000001d4cd.snap
│       └── wal
│           ├── 0000000000000000-0000000000000000.wal
│           └── 0000000000000001-0000000000012017.wal
├── etcdctl
└── run.sh

其中run.sh是启动脚本，内容如下：

killall -9 etcd
./etcd \
-name etcd0 \
-data-dir etcd0.etcd \
-initial-advertise-peer-urls http://master:2380 \
-listen-peer-urls http://master:2380 \
-listen-client-urls http://master:2379,http://127.0.0.1:2379 \
-advertise-client-urls http://master:2379 \
-initial-cluster-token etcd-cluster \
-initial-cluster etcd0=http://master:2380,etcd1=http://dockertest4:2380,etcd2=http://dockertest5:2380 \
-initial-cluster-state new

（2）minion1的etch配置 （放在/home/docker/xu/etcd目录下）
root@dockertest4:/home/docker/xu/etcd# tree
.
├── etcd
├── etcd1.etcd
│   └── member
│       ├── snap
│       │   ├── 000000000000005c-0000000000013888.snap
│       │   ├── 0000000000000085-0000000000015f99.snap
│       │   ├── 00000000000000bc-00000000000186aa.snap
│       │   ├── 00000000000000bf-000000000001adbb.snap
│       │   └── 00000000000000bf-000000000001d4cd.snap
│       └── wal
│           ├── 0000000000000000-0000000000000000.wal
│           └── 0000000000000001-0000000000012025.wal
├── etcdctl
└── run.sh

其中run.sh是启动脚本，内容如下：

killall -9 etcd
./etcd \
-name etcd1 \
-data-dir etcd1.etcd \
-initial-advertise-peer-urls http://dockertest4:2380 \
-listen-peer-urls http://dockertest4:2380 \
-listen-client-urls http://dockertest4:2379,http://127.0.0.1:2379 \
-advertise-client-urls http://dockertest4:2379 \
-initial-cluster-token etcd-cluster \
-initial-cluster etcd0=http://master:2380,etcd1=http://dockertest4:2380,etcd2=http://dockertest5:2380 \
-initial-cluster-state new

（3）minion2的etcd配置 （放在/home/docker/xu/etcd目录下）

root@dockertest5:/home/docker/xu/etcd# tree
.
├── etcd
├── etcd2.etcd
│   └── member
│       ├── snap
│       │   ├── 000000000000005c-0000000000013889.snap
│       │   ├── 0000000000000085-0000000000015f9a.snap
│       │   ├── 00000000000000bc-00000000000186ab.snap
│       │   ├── 00000000000000bf-000000000001adbc.snap
│       │   └── 00000000000000bf-000000000001d4cd.snap
│       └── wal
│           ├── 0000000000000000-0000000000000000.wal
│           └── 0000000000000001-000000000001200