《kubernetes-1.8.0》02-etcd群集搭建

《kubernetes-1.8.0》02-etcd群集搭建

—《kubernetes 1.8.0 测试环境安装部署》

— 时间:2017-11-22

一、etcd软件安装

通过在node-131、node-132、node-133上yum安装,也可使用mritd提供tarball中的etcd rpm包。

yum install -y etcd 

当前版本为”etcd-3.2.701.el7”

二、分发etcd相关证书

分发证书

$ cd ~/etcd_ssl

$ for IP in `seq 131 133`;do
    ssh root@172.18.169.$IP mkdir /etc/etcd/ssl
    scp *.pem root@172.18.169.$IP:/etc/etcd/ssl
    ssh root@172.18.169.$IP chown -R etcd:etcd /etc/etcd/ssl
    ssh root@172.18.169.$IP chmod -R 644 /etc/etcd/ssl/*
    ssh root@172.18.169.$IP chmod 755 /etc/etcd/ssl
done

设置etcd数据目录owner/group

for IP in `seq 131 133`;do
    ssh root@172.18.169.$IP chown -R etcd:etcd /var/lib/etcd
done

三、修改etcd配置文件

$ vim /etc/etcd/etcd.conf

示例:

# [member]
ETCD_NAME=node-131
ETCD_DATA_DIR="/var/lib/etcd/node-131.etcd"
ETCD_WAL_DIR="/var/lib/etcd/wal"
ETCD_SNAPSHOT_COUNT="100"
ETCD_HEARTBEAT_INTERVAL="100"
ETCD_ELECTION_TIMEOUT="1000"
ETCD_LISTEN_PEER_URLS="https://172.18.169.131:2380"
ETCD_LISTEN_CLIENT_URLS="https://172.18.169.131:2379,http://127.0.0.1:2379"
ETCD_MAX_SNAPSHOTS="5"
ETCD_MAX_WALS="5"
#ETCD_CORS=""

# [cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://172.18.169.131:2380"
# if you use different ETCD_NAME (e.g. test), set ETCD_INITIAL_CLUSTER value for this name, i.e. "test=http://..."
ETCD_INITIAL_CLUSTER="node-131=https://172.18.169.131:2380,node-132=https://172.18.169.132:2380,node-133=https://172.18.169.133:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="https://172.18.169.131:2379"
#ETCD_DISCOVERY=""
#ETCD_DISCOVERY_SRV=""
#ETCD_DISCOVERY_FALLBACK="proxy"
#ETCD_DISCOVERY_PROXY=""
#ETCD_STRICT_RECONFIG_CHECK="false"
#ETCD_AUTO_COMPACTION_RETENTION="0"

# [proxy]
#ETCD_PROXY="off"
#ETCD_PROXY_FAILURE_WAIT="5000"
#ETCD_PROXY_REFRESH_INTERVAL="30000"
#ETCD_PROXY_DIAL_TIMEOUT="1000"
#ETCD_PROXY_WRITE_TIMEOUT="5000"
#ETCD_PROXY_READ_TIMEOUT="0"

# [security]
ETCD_CERT_FILE="/etc/etcd/ssl/etcd.pem"
ETCD_KEY_FILE="/etc/etcd/ssl/etcd-key.pem"
ETCD_CLIENT_CERT_AUTH="true"
ETCD_TRUSTED_CA_FILE="/etc/etcd/ssl/etcd-root-ca.pem"
ETCD_AUTO_TLS="true"
ETCD_PEER_CERT_FILE="/etc/etcd/ssl/etcd.pem"
ETCD_PEER_KEY_FILE="/etc/etcd/ssl/etcd-key.pem"
ETCD_PEER_CLIENT_CERT_AUTH="true"
ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ssl/etcd-root-ca.pem"
ETCD_PEER_AUTO_TLS="true"

# [logging]
#ETCD_DEBUG="false"
# examples for -log-package-levels etcdserver=WARNING,security=DEBUG
#ETCD_LOG_PACKAGE_LEVELS=""
  • ETCD_NAME: etcd节点名称,如果是静态etcd cluster,必须与ETCD_INITIAL_CLUSTER中的名称进行对应。
  • ETCD_INITIAL_CLUSTER_STATE: new为新建集群,如果是加入一个已经存在的etcd集群,需将该参数改为existing
  • ETCD_DATA_DIR=:存放etcd member等db数据
  • ETCD_CLIENT_CERT_AUTH、ETCD_TRUSTED_CA_FILE、ETCD_CERT_FILE、ETCD_KEY_FILE等:为etcd TLS所需证书,制定之前创建的证书即可。
  • 更多etcd TLS 选项查看 etcd 3.2.7 document

node-132、node-133修改对应参数。

三、启动etcd服务并验证

node-131、node-132、node-133上执行:

systemctl daemon-reload
systemctl start etcd
systemctl enable etcd

检查节点状态:

$ export ETCDCTL_API=3
$ etcdctl --cacert=/etc/etcd/ssl/etcd-root-ca.pem \
--cert=/etc/etcd/ssl/etcd.pem \
--key=/etc/etcd/ssl/etcd-key.pem \
--endpoints=https://172.18.169.131:2379,https://172.18.169.132:2379,https://172.18.169.133:2379 \
endpoint health
https://172.18.169.133:2379 is healthy: successfully committed proposal: took = 2.016793ms
https://172.18.169.132:2379 is healthy: successfully committed proposal: took = 2.005839ms
https://172.18.169.131:2379 is healthy: successfully committed proposal: took = 1.167565ms

检查etcd 版本:

[root@node-131 etcd_ssl]# etcdctl version
etcdctl version: 3.2.7
API version: 3.2

加上TLS之后 etcd api verison 自动切换成了3.2。原先的cluster-health ls pwd 什么的都不管用了。

四、删除、添加etcd节点

查看群集成员:

$ export ETCDCTL_API=3
$ etcdctl --cacert=/etc/etcd/ssl/etcd-root-ca.pem \
--cert=/etc/etcd/ssl/etcd.pem \
--key=/etc/etcd/ssl/etcd-key.pem \
--endpoints=https://172.18.169.131:2379,https://172.18.169.132:2379,https://172.18.169.133:2379 \
member list
5d5554b1f11aba62, started, node-131, https://172.18.169.131:2380, https://172.18.169.131:2379
8b10a60fc4b98fcb, started, node-133, https://172.18.169.133:2380, https://172.18.169.133:2379
cd1bf9a8ae65b314, started, node-132, https://172.18.169.132:2380, https://172.18.169.132:2379

删除note-133:

$ etcdctl --cacert=/etc/etcd/ssl/etcd-root-ca.pem \
--cert=/etc/etcd/ssl/etcd.pem \
--key=/etc/etcd/ssl/etcd-key.pem \
--endpoints=https://172.18.169.131:2379,https://172.18.169.132:2379,https://172.18.169.133:2379 \
member remove 8b10a60fc4b98fcb

Member 8b10a60fc4b98fcb removed from cluster 3697c33650b7b984

$ etcdctl --cacert=/etc/etcd/ssl/etcd-root-ca.pem \
--cert=/etc/etcd/ssl/etcd.pem \
--key=/etc/etcd/ssl/etcd-key.pem \
--endpoints=https://172.18.169.131:2379,https://172.18.169.132:2379,https://172.18.169.133:2379 \
member list

5d5554b1f11aba62, started, node-131, https://172.18.169.131:2380, https://172.18.169.131:2379
cd1bf9a8ae65b314, started, node-132, https://172.18.169.132:2380, https://172.18.169.132:2379

将node-133添加回etcd群集:

群集节点添加member:

etcdctl --cacert=/etc/etcd/ssl/etcd-root-ca.pem \
--cert=/etc/etcd/ssl/etcd.pem \
--key=/etc/etcd/ssl/etcd-key.pem \
--endpoints=https://172.18.169.131:2379,https://172.18.169.132:2379,https://172.18.169.133:2379 \
member add node-133 \
--peer-urls=https://172.18.169.133:2380 

Member 17948fc49f73cbb9 added to cluster 3697c33650b7b984

ETCD_NAME="node-133"
ETCD_INITIAL_CLUSTER="node-133=https://172.18.169.133:2380,node-131=https://172.18.169.131:2380,node-132=https://172.18.169.132:2380"
ETCD_INITIAL_CLUSTER_STATE="existing"
  • --peer-urls:api 3.2后加member需要增加这个选项
    修改member设置
##清空member上的数据目录
$ sudo systemctl start etcd
$ sudo rm -rf /var/lib/etcd/*

##修改member etcd配置文件修改
ETCD_INITIAL_CLUSTER_STATE="existing"

##启动服务
$ sudo systemctl start etcd

验证节点是否加入:

$ sudo etcdctl \
--cacert=/etc/etcd/ssl/etcd-root-ca.pem \
--cert=/etc/etcd/ssl/etcd.pem \
--key=/etc/etcd/ssl/etcd-key.pem \
--endpoints=https://172.18.169.131:2379,https://172.18.169.132:2379,https://172.18.169.133:2379 \
member list

17948fc49f73cbb9, started, node-133, https://172.18.169.133:2380, https://172.18.169.133:2379
5d5554b1f11aba62, started, node-131, https://172.18.169.131:2380, https://172.18.169.131:2379
cd1bf9a8ae65b314, started, node-132, https://172.18.169.132:2380, https://172.18.169.132:2379

$ sudo etcdctl \
--cacert=/etc/etcd/ssl/etcd-root-ca.pem \
--cert=/etc/etcd/ssl/etcd.pem \
--key=/etc/etcd/ssl/etcd-key.pem \
--endpoints=https://172.18.169.131:2379,https://172.18.169.132:2379,https://172.18.169.133:2379 \
endpoint health

https://172.18.169.133:2379 is healthy: successfully committed proposal: took = 2.348909ms
https://172.18.169.132:2379 is healthy: successfully committed proposal: took = 2.139596ms
https://172.18.169.131:2379 is healthy: successfully committed proposal: took = 1.222221ms

至此etcd TLS集群搭建完成

本系列其他内容:

参考链接:

https://mritd.me/2017/10/09/set-up-kubernetes-1.8-ha-cluster/

https://github.com/opsnull/follow-me-install-kubernetes-cluster

https://coreos.com/etcd/docs/3.2.7/index.html

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值