【运维】K8S集群部署系列之ETCD集群搭建（一）

ETCD 普通集群搭建

将从本文开始记录K8S集群部署的全部操作，打造傻瓜式部署教程。

简介

ETCD是基于GO语言编写的用于配置共享和服务发现的键值存储系统，以简单、安全、快速、可信为核心。

系统环境

Node	IP	CentOS	kernel	cpu	memory
master	192.168.159.3	CentOS Linux release 7.4.1708 (Core)	3.10.0-693.el7.x86_64	Intel® Core™ i5-7500 CPU @ 3.40GHz * 1	2G
node1	192.168.159.4	CentOS Linux release 7.4.1708 (Core)	3.10.0-693.el7.x86_64	Intel® Core™ i5-7500 CPU @ 3.40GHz * 1	2G
node2	192.168.159.5	CentOS Linux release 7.4.1708 (Core)	3.10.0-693.el7.x86_64	Intel® Core™ i5-7500 CPU @ 3.40GHz * 1	2G
node3	192.168.159.6	CentOS Linux release 7.4.1708 (Core)	3.10.0-693.el7.x86_64	Intel® Core™ i5-7500 CPU @ 3.40GHz * 1	2G

软件环境

Node	IP	etcd
master	192.168.159.3	3.3.13
node1	192.168.159.4	3.3.13
node2	192.168.159.5	3.3.13

etcd安装

ETCD官方文档:https://etcd.io/docs/v3.3.12/op-guide/configuration/

下载安装包

• 下载地址

  https://github.com/etcd-io/etcd/releases

• 下载安装包

  在四个节点执行以下操作

    mkdir /home/k8s
    cd /home/k8s
    wget https://github.com/etcd-io/etcd/releases/download/v3.3.13/etcd-v3.3.13-linux-amd64.tar.gz
    tar -zxvf etcd-v3.3.13-linux-amd64.tar.gz
    mv etcd-v3.3.13-linux-amd64 etcd
    chmod -R +x etcd/
    cp -f ./{etcd,etcdctl} /usr/bin/
    cp -f ./{etcd,etcdctl} /usr/local/bin/
  

集群节点配置文件

master节点配置

mkdir -p /opt/etcd/{data,etc}
cat > /opt/etcd/etc/etcd.conf << EOF
#[Member]
ETCD_NAME="etcd-1"
ETCD_DATA_DIR="/opt/etcd/data"
ETCD_LISTEN_PEER_URLS="http://192.168.159.3:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.159.3:2379,http://127.0.0.1:2379"

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.159.3:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.159.3:2379"
ETCD_INITIAL_CLUSTER="etcd-1=http://192.168.159.3:2380,etcd-2=http://192.168.159.4:2380,etcd-3=http://192.168.159.5:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF

• 名词解释

  	ETCD_NAME： 节点名称
  	ETCD_DATA_DIR: 节点数据目录
  	ETCD_LISTEN_PEER_URLS： 节点服务端通信地址
  	ETCD_LISTEN_CLIENT_URLS： 节点客户端通信地址
  	
  	ETCD_INITIAL_ADVERTISE_PEER_URLS： 节点集群内服务端通信地址
  	ETCD_ADVERTISE_CLIENT_URLS： 节点集群内客户端通信地址
  	ETCD_INITIAL_CLUSTER： 集群节点集
  	ETCD_INITIAL_CLUSTER_TOKEN： token，标识同一集群
  	ETCD_INITIAL_CLUSTER_STATE： new为新建集群，existing为加入已创建集群，此处为new
  

node1节点配置

mkdir -p /opt/etcd/{data,etc}
cat > /opt/etcd/etc/etcd.conf << EOF
#[Member]
ETCD_NAME="etcd-2"
ETCD_DATA_DIR="/opt/etcd/data"
ETCD_LISTEN_PEER_URLS="http://192.168.159.4:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.159.4:2379,http://127.0.0.1:2379"

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.159.4:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.159.4:2379"
ETCD_INITIAL_CLUSTER="etcd-1=http://192.168.159.3:2380,etcd-2=http://192.168.159.4:2380,etcd-3=http://192.168.159.5:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF

node2节点配置

mkdir -p /opt/etcd/{data,etc}
cat > /opt/etcd/etc/etcd.conf << EOF
#[Member]
ETCD_NAME="etcd-3"
ETCD_DATA_DIR="/opt/etcd/data"
ETCD_LISTEN_PEER_URLS="http://192.168.159.5:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.159.5:2379,http://127.0.0.1:2379"

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.159.5:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.159.5:2379"
ETCD_INITIAL_CLUSTER="etcd-1=http://192.168.159.3:2380,etcd-2=http://192.168.159.4:2380,etcd-3=http://192.168.159.5:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF

ETCD服务启动文件

systemd中文手册:http://www.jinbuguo.com/systemd/systemd.exec.html

cat >  vim /usr/lib/systemd/system/etcd.service << EOF    
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=-/opt/etcd/etc/etcd.conf
ExecStart=/home/k8s/etcd/etcd \
--name=${ETCD_NAME} \
--data-dir=${ETCD_DATA_DIR} \
--listen-peer-urls=${ETCD_LISTEN_PEER_URLS} \
--listen-client-urls=${ETCD_LISTEN_CLIENT_URLS} \
--initial-advertise-peer-urls=${ETCD_INITIAL_ADVERTISE_PEER_URLS} \
--advertise-client-urls=${ETCD_ADVERTISE_CLIENT_URLS} \
--initial-cluster=${ETCD_INITIAL_CLUSTER} \
--initial-cluster-token=${ETCD_INITIAL_CLUSTER_TOKEN} \
--initial-cluster-state=${ETCD_INITIAL_CLUSTER_STATE} 
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

网络准备

cat >>/etc/hosts<< EOF
    192.168.159.3 master
    192.168.159.4 node1
    192.168.159.5 node2
    192.168.159.6 node3
EOF
# 防火墙设置，开放2379和2380端口,如果启动防火墙但未放开端口则集群状态为“degraded”，
# 开启防火墙的节点状态为“are all unreachable”
# 2379端口提供给客户端访问集群，客户端如：etcdctl
# 2380端口提供给集群节点间通信
systemctl start firewalld
firewall-cmd --zone=public --add-port=2379/tcp --permanent
firewall-cmd --zone=public --add-port=2380/tcp --permanent
firewall-cmd --reload
firewall-cmd --list-all

启动集群

systemctl start etcd

注意: 集群节点需要同时启动才能成功，否则会报错某个节点无法找到；如果是重新创建集群，则需要删除旧的数据目录。

集群健康检查

etcdctl cluster-health

    member 8ada33a16cb8b5f9 is healthy: got healthy result from http://192.168.159.4:2379
    member df5c33b8666738a6 is healthy: got healthy result from http://192.168.159.3:2379
    member e689a191b9fab04f is healthy: got healthy result from http://192.168.159.5:2379
    cluster is healthy # cluster is degraded表示集群至少有一个节点不可达

etcdctl member list

    8ada33a16cb8b5f9: name=etcd-2 peerURLs=http://192.168.159.4:2380 clientURLs=http://192.168.159.4:2379 isLeader=true # 主节点
    df5c33b8666738a6: name=etcd-1 peerURLs=http://192.168.159.3:2380 clientURLs=http://192.168.159.3:2379 isLeader=false
    e689a191b9fab04f: name=etcd-3 peerURLs=http://192.168.159.5:2380 clientURLs=http://192.168.159.5:2379 isLeader=false

至此ETCD数据库普通集群搭建完毕；
下一篇将介绍TLS证书的制作。