k8s 1.9.8 安装（3）etcd集群安装、Flannel网络搭建

返回总目录连接

1.etcd集群安装（三个节点都需要安装，只是etcd.conf需要注意IP和名字）

1.1 资源下载
git上下载安装包，也可以直接使用yum安装

    yum install -y etcd
    git下载地址，我选择了etcd-v3.1.10-linux-amd64
    https://github.com/etcd-io/etcd/tags
    https://github.com/etcd-io/etcd/releases/download/v3.1.10/etcd-v3.1.10-linux-amd64.tar.gz
    cd /usr/local/src && wget https://github.com/etcd-io/etcd/releases/download/v3.1.10/etcd-v3.1.10-linux-amd64.tar.gz 

1.2解压资源，把etcd启动脚本复制到/usr/bin

tar -zxvf etcd-v3.1.10-linux-amd64.tar.gz && cp etcd-v3.1.10-linux-amd64/{etcd,etcdctl}  /usr/bin

1.3配置etcd.service

vim /usr/lib/systemd/system/etcd.service 
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/coreos
 
[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
ExecStart=/usr/bin/etcd \
  --name ${ETCD_NAME} \
  --cert-file=/etc/kubernetes/ssl/kubernetes.pem \
  --key-file=/etc/kubernetes/ssl/kubernetes-key.pem \
  --peer-cert-file=/etc/kubernetes/ssl/kubernetes.pem \
  --peer-key-file=/etc/kubernetes/ssl/kubernetes-key.pem \
  --trusted-ca-file=/etc/kubernetes/ssl/ca.pem \
  --peer-trusted-ca-file=/etc/kubernetes/ssl/ca.pem \
  --initial-advertise-peer-urls ${ETCD_INITIAL_ADVERTISE_PEER_URLS} \
  --listen-peer-urls ${ETCD_LISTEN_PEER_URLS} \
  --listen-client-urls ${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 \
  --advertise-client-urls ${ETCD_ADVERTISE_CLIENT_URLS} \
  --initial-cluster-token ${ETCD_INITIAL_CLUSTER_TOKEN} \
  --initial-cluster etcd0=https://master:2380,etcd1=https://node1:2380,etcd2=https://node2:2380 \
  --initial-cluster-state new \
  --data-dir=${ETCD_DATA_DIR}
Restart=on-failure
RestartSec=5
LimitNOFILE=65536

1.4配置etcd.conf

注意IP和名字ETCD_NAME不同

vim  /etc/etcd/etcd.conf 
# [member]
ETCD_NAME=etcd0
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_LISTEN_PEER_URLS="https://master:2380"
ETCD_LISTEN_CLIENT_URLS="https://master:2379"
 
#[cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://master:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="https://master:2379"
node1节点
vim /etc/etcd/etcd.conf
# [member]
ETCD_NAME=etcd1
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_LISTEN_PEER_URLS="https://node1:2380"
ETCD_LISTEN_CLIENT_URLS="https://node1:2379"
 node2节点
#[cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://node1:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="https://node1:2379"

vim /etc/etcd/etcd.conf
# [member]
ETCD_NAME=etcd1
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_LISTEN_PEER_URLS="https://node2:2380"
ETCD_LISTEN_CLIENT_URLS="https://node2:2379"
 
#[cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://node2:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="https://node2:2379"

1.5启动、设置为开机启动（三台机器同时启动）

systemctl daemon-reload && systemctl start etcd
systemctl enable etcd

1.6检查健康情况

  etcdctl \
  --ca-file=/etc/kubernetes/ssl/ca.pem \
  --cert-file=/etc/kubernetes/ssl/kubernetes.pem \
  --key-file=/etc/kubernetes/ssl/kubernetes-key.pem \
  cluster-health

2.Flannel 网络安装(只有node节点需要配置)

2.1yum安装flannel

yum -y install flannel

2.2配置flanneld

vim /etc/sysconfig/flanneld


FLANNEL_ETCD_ENDPOINTS="https://master:2379,https://node1:2379,https://node2:2379"

FLANNEL_ETCD_PREFIX="/kubernetes/network"

FLANNEL_OPTIONS="-etcd-cafile=/etc/kubernetes/ssl/ca.pem -etcd-certfile=/etc/kubernetes/ssl/kubernetes.pem -etcd-keyfile=/etc/kubernetes/ssl/kubernetes-key.pem"

2.3配置flanneld.service

vim /usr/lib/systemd/system/flanneld.service 
[Unit]
Description=Flanneld overlay address etcd agent
After=network.target
After=network-online.target
Wants=network-online.target
After=etcd.service
Before=docker.service
 
[Service]
Type=notify
EnvironmentFile=/etc/sysconfig/flanneld
EnvironmentFile=-/etc/sysconfig/docker-network
ExecStart=/usr/bin/flanneld-start \
  -etcd-endpoints=${FLANNEL_ETCD_ENDPOINTS} \
  -etcd-prefix=${FLANNEL_ETCD_PREFIX} \
  $FLANNEL_OPTIONS
ExecStartPost=/usr/libexec/flannel/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/docker
Restart=on-failure
 
[Install]
WantedBy=multi-user.target
RequiredBy=docker.service

2.4etcd创建网络

 etcdctl \
  --ca-file=/etc/kubernetes/ssl/ca.pem \
  --cert-file=/etc/kubernetes/ssl/kubernetes.pem \
  --key-file=/etc/kubernetes/ssl/kubernetes-key.pem \
  mk  /kubernetes/network/config  '{"Network":"172.17.0.0/16"}' 

2.5安装docker
2.5.1 添加yum源

添加yum源
cat >/etc/yum.repos.d/docker.repo <<-EOF            
    [dockerrepo]
	name=Docker Repository
	baseurl=https://yum.dockerproject.org/repo/main/centos/7
	enabled=1
	gpgcheck=1
	gpgkey=https://yum.dockerproject.org/gpg
EOF

2.5.2 安装

yum install -y docker-selinux
yum install -y docker-engine

2.5.3配置

vim /usr/lib/systemd/system/docker.service

[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.io
 
[Service]
Environment="PATH=/usr/local/bin:/bin:/sbin:/usr/bin:/usr/sbin"
EnvironmentFile=-/run/flannel/subnet.env
EnvironmentFile=-/run/flannel/docker
ExecStart=/usr/bin/dockerd \
  --exec-opt native.cgroupdriver=cgroupfs \
  --log-level=error \
  --log-driver=json-file \
  --storage-driver=overlay \
  $DOCKER_NETWORK_OPTIONS
ExecStartPost=/sbin/iptables -I FORWARD -s 0.0.0.0/0 -j ACCEPT
ExecReload=/bin/kill -s HUP $MAINPID
Restart=on-failure
RestartSec=5
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
Delegate=yes
KillMode=process
 
[Install]
WantedBy=multi-user.target

2.5.4启动开机启动

systemctl enable docker.service 
systemctl start docker.service

2.6启动flanneld，将flanneld设置为开机启动

systemctl daemon-reload && systemctl start flanneld
systemctl enable flanneld

2.7查看flannel

etcdctl \
  --ca-file=/etc/kubernetes/ssl/ca.pem \
  --cert-file=/etc/kubernetes/ssl/kubernetes.pem \
  --key-file=/etc/kubernetes/ssl/kubernetes-key.pem \
   ls -r |grep subnets
结果
/kubernetes/network/subnets
/kubernetes/network/subnets/172.17.14.0-24
/kubernetes/network/subnets/172.17.62.0-24
/kubernetes/network/subnets/172.17.39.0-24

2.8flannel网络原理
ifconfig可以查看docker的网卡和flannel网卡