理解Kubernetes(1):手工搭建Kubernetes测试环境
理解Kubernetes系列文章:
- 手工搭建环境
- 基本概念和操作
1. 基础环境准备
准备 3个Ubuntu节点,操作系统版本为 16.04,并做好以下配置:
- 系统升级
- 设置 /etc/hosts 文件,保持一致
- 设置从 0 节点上无密码ssh 其它两个节点
节点名称 | IP地址 | etcd | flanneld | K8S | docker |
kub-node-0 | 172.23.100.4 | Y | Y |
master:
kubctl
kube-apiserver
kuber-controller
kuber-scheduler
|
Y |
kub-node-1 | 172.23.100.5 | Y | Y | node: kube-proxy kubelet |
Y |
kub-node-2 | 172.23.100.6 | Y | Y | node: kube-proxy kubelet |
Y |
2. 安装与部署
2.1 安装 etcd
2.1.1 安装
在3个节点上运行以下命令来安装 etcd 3.2.5 版本:
ETCD_VERSION=${ETCD_VERSION:-"3.2.5"} ETCD="etcd-v${ETCD_VERSION}-linux-amd64" curl -L https://github.com/coreos/etcd/releases/download/v${ETCD_VERSION}/${ETCD}.tar.gz -o etcd.tar.gz tar xzf etcd.tar.gz -C /tmp mv /tmp/etcd-v${ETCD_VERSION}-linux-amd64 /opt/bin/
2.1.2 配置
在3个节点上做如下配置:
- 创建目录:
sudo mkdir -p /var/lib/etcd/ sudo mkdir -p /opt/config/
- 创建 /opt/config/etcd.conf 文件:
ETCD_DATA_DIR=/var/lib/etcd ETCD_NAME="kub-node-0" ETCD_INITIAL_CLUSTER="kub-node-0=http://172.23.100.4:2380,kub-node-1=http://172.23.100.5:2380,kub-node-2=http://172.23.100.6:2380" ETCD_INITIAL_CLUSTER_STATE=new ETCD_LISTEN_PEER_URLS=http://172.23.100.4:2380 ETCD_INITIAL_ADVERTISE_PEER_URLS=http://172.23.100.4:2380 ETCD_ADVERTISE_CLIENT_URLS=http://172.23.100.4:2379 ETCD_LISTEN_CLIENT_URLS=http://172.23.100.4:2379,http://127.0.0.1:2379
注意:
(1)在 0 节点上 etcd cluster 起来后,1 和 2 上的 ETCD_INITIAL_CLUSTER_STATE 值需要改成 existing,表示加入已有集群。否则的话,它自己会创建一个cluster,而不是加入已有cluster。
(2)在每个节点上,IP 地址需要修改为本机地址。
- 创建 /lib/systemd/system/etcd.service 文件:
[Unit] Description=Etcd Server Documentation=https://github.com/coreos/etcd After=network.target [Service] User=root Type=simple EnvironmentFile=-/opt/config/etcd.conf ExecStart=/opt/bin/etcd Restart=on-failure RestartSec=10s LimitNOFILE=40000 [Install] WantedBy=multi-user.target
每个节点上都是一样的。
- 在三个节点上启动服务:
systemctl daemon-reload systemctl enable etcd systemctl start etcd
2.1.3 测试服务
- 查看etcd集群状态:
root@kub-node-2:/home/ubuntu# /opt/bin/etcdctl cluster-health member 664b85ff39242fbc is healthy: got healthy result from http://172.23.100.6:2379 member 9dd263662a4b6f73 is healthy: got healthy result from http://172.23.100.4:2379 member b17535572fd6a37b is healthy: got healthy result from http://172.23.100.5:2379 cluster is healthy
- 查看 etcd 集群成员:
root@kub-node-0:/home/ubuntu# /opt/bin/etcdctl member list 9dd263662a4b6f73: name=kub-node-0 peerURLs=http://172.23.100.4:2380 clientURLs=http://172.23.100.4:2379 isLeader=false b17535572fd6a37b: name=kub-node-1 peerURLs=http://172.23.100.5:2380 clientURLs=http://172.23.100.5:2379 isLeader=true e6db3cac1db23670: name=kub-node-2 peerURLs=http://172.23.100.6:2380 clientURLs=http://172.23.100.6:2379 isLeader=false
2.2 部署flanneld
2.2.1 安装 0.8.0 版本
在每个节点上:
curl -L https://github.com/coreos/flannel/releases/download/v0.8.0/flannel-v0.8.0-linux-amd64.tar.gz flannel.tar.gz tar xzf flannel.tar.gz -C /tmp mv /tmp/flanneld /opt/bin/
2.2.2 配置
在每个节点上:
- 创建 /lib/systemd/system/flanneld.service 文件:
[Unit] Description=Flanneld Documentation=https://github.com/coreos/flannel After=network.target Before=docker.service [Service] User=root ExecStart=/opt/bin/flanneld \ --etcd-endpoints="http://172.23.100.4:2379,http://172.23.100.5:2379,http://172.23.100.4:2379" \ --iface=172.23.100.4 \ --ip-masq Restart=on-failure Type=notify LimitNOFILE=65536
注意:在每个节点上,iface 设置为本机ip。
- 在 0 node上,运行
/opt/bin/etcdctl --endpoints="http://172.23.100.4:2379,http://172.23.100.5:2379,http://172.23.100.4:2379" mk /coreos.com/network/config \ '{"Network":"10.1.0.0/16", "Backend": {"Type": "vxlan"}}'
确认:
root@kub-node-0:/home/ubuntu# /opt/bin/etcdctl --endpoints="http://172.23.100.4:2379,http://172.23.100.5:2379,http://172.23.100.4:2379" get /coreos.com/network/config { "Network":"10.1.0.0/16", "Backend": { "Type": "vxlan"}}
- 在三个节点上启动 flannled:
systemctl daemon-reload systemctl enable flanneld systemctl start flanneld
备注:flannel服务需要先于Docker启动。flannel服务启动时主要做了以下几步的工作:
- 从etcd中获取network的配置信息。
- 划分subnet,并在etcd中进行注册。
- 将子网信息记录到/run/flannel/subnet.env中。
此时,能看到 etcd 中的 subnet:
root@kub-node-0:/home/ubuntu/kub# /opt/bin/etcdctl --endpoints="http://172.23.100.4:2379,http://172.23.100.5:2379,http://172.23.100.4:2379"; ls /coreos.com/network/subnets /coreos.com/network/subnets/10.1.35.0-24 /coreos.com/network/subnets/10.1.1.0-24 /coreos.com/network/subnets/10.1.79.0-24
2.2.3 验证
- 通过运行 service flanneld status 查看其状态。
- 检查 flannel 虚拟网卡。它们的配置需要和 etcd 中的配置一致。
root@kub-node-0:/home/ubuntu/kub# ifconfig flannel.1 flannel.1 Link encap:Ethernet HWaddr 22:fc:69:01:33:30 inet addr:10.1.35.0 Bcast:0.0.0.0 Mask:255.255.255.255