文章目录
1 机器环境
1.1 机器1
hostname:k8s-master1
ip:172.16.159.130
1.2 机器2
hostname:k8s-node1
ip:172.16.159.131
1.3 机器3
hostname:k8s-node2
ip:172.16.159.132
2 软件介绍
2.1 操作系统
centos:CentOS Linux release 7.5.1804
内核:3.10.0-862.el7.x86_64
2.2 Docker
docker:docker-18.03.1-ce.tgz
2.3 etcd
etcd:etcd-v3.2.9-linux-amd64.tar.gz
2.4 flanneld
flanneld:flannel-v0.7.1-linux-amd64.tar.gz
3 安装步骤
3.1 安装和配置 docker(master和node节点部署)
k8s-master1,k8s-node1,k8s-node2都得安装docker。
3.1.1 下载最新的 docker 二进制文件
# mkdir -p /Data/apps/docker
# cd /Data/apps/docker
# wget https://download.docker.com/linux/static/stable/x86_64/docker-18.03.1-ce.tgz
# tar -xvf docker-18.03.1-ce.tgz
# cp docker/docker* /usr/bin
3.1.2 创建 docker 的 启动文件
vi /usr/lib/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.io
[Service]
Environment="PATH=/root/local/bin:/bin:/sbin:/usr/bin:/usr/sbin"
EnvironmentFile=-/run/flannel/docker
ExecStart=/usr/bin/dockerd --log-level=error $DOCKER_NETWORK_OPTIONS
ExecReload=/bin/kill -s HUP $MAINPID
Restart=on-failure
RestartSec=5
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
Delegate=yes
KillMode=process
[Install]
WantedBy=multi-user.target
3.1.3 启动 dockerd
# systemctl daemon-reload
# systemctl stop firewalld
# systemctl disable firewalld
# iptables -F && iptables -X && iptables -F -t nat && iptables -X -t nat
# systemctl enable docker
# systemctl start docker
3.1.4 检查 docker 服务
# docker version
Client:
Version: 18.03.1-ce
API version: 1.37
Go version: go1.9.2
Git commit: 9ee9f40
Built: Thu Apr 26 07:12:25 2018
OS/Arch: linux/amd64
Experimental: false
Orchestrator: swarm
Server:
Engine:
Version: 18.03.1-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.9.5
Git commit: 9ee9f40
Built: Thu Apr 26 07:23:03 2018
OS/Arch: linux/amd64
Experimental: false
3.2 安装和配置 etcd(master节点部署)
k8s-master1安装etcd-v3.2.9-linux-amd64.tar.gz
3.2.1 etcd数据库安装
# mkdir -p /Data/apps/etcd
# cd /Data/apps/etcd
# wget https://github.com/coreos/etcd/releases/download/v3.2.9/etcd-v3.2.9-linux-amd64.tar.gz
# tar -xvf etcd-v3.2.9-linux-amd64.tar.gz
# cd etcd-v3.2.9-linux-amd64
# cp etcd etcdctl /usr/bin/
3.2.2 创建etcd作目录
# mkdir -p /var/lib/etcd
3.2.3 创建 etcd 的 启动文件
vi /etc/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
# set GOMAXPROCS to number of processors
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/bin/etcd --name=\"${ETCD_NAME}\" --data-dir=\"${ETCD_DATA_DIR}\" --listen-client-urls=\"${ETCD_LISTEN_CLIENT_URLS}\""
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
3.2.4 创建配置/etc/etcd/etcd.conf文件
mkdir -p /etc/etcd/
vi /etc/etcd/etcd.conf
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
ETCD_NAME="default"
ETCD_ADVERTISE_CLIENT_URLS="http://0.0.0.0:2379"```
3.2.5 配置开机启动
# systemctl daemon-reload
# systemctl enable etcd.service
# systemctl start etcd.service
3.2.6 检验etcd是否安装成功
# etcdctl cluster-health
member 8e9e05c52164694d is healthy: got healthy result from http://0.0.0.0:2379
cluster is healthy
3.3 安装和配置 kubernetes(master节点部署)
k8s-master1安装kubernetes。
3.3.1 安装kubernetes 二进制文件
# mkdir -p /Data/apps/k8s
# cd /Data/apps/k8s
# wget https://dl.k8s.io/v1.11.1/kubernetes-server-linux-amd64.tar.gz
# tar -xzvf kubernetes-server-linux-amd64.tar.gz
# cd kubernetes/server/bin/
# cp kube-apiserver kube-controller-manager kube-scheduler kubectl /usr/bin/
3.3.2 配置kube-apiserver.service
1 配置kube-apiserver.service启动项目
vi /usr/lib/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target
After=etcd.service
[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/apiserver
ExecStart=/usr/bin/kube-apiserver \
$KUBE_LOGTOSTDERR \
$KUBE_LOG_LEVEL \
$KUBE_ETCD_SERVERS \
$KUBE_API_ADDRESS \
$KUBE_API_PORT \
$KUBELET_PORT \
$KUBE_ALLOW_PRIV \
$KUBE_SERVICE_ADDRESSES \
$KUBE_ADMISSION_CONTROL \
$KUBE_API_ARGS
Restart=on-failure
Type=notify
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
2 配置kube-apiserver.service参数
mkdir -p /etc/kubernetes/
vi /etc/kubernetes/apiserver
KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"
KUBE_API_PORT="--port=8080"
KUBE_ETCD_SERVERS="--etcd-servers=http://172.16.159.130:2379"
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=172.17.0.1/16"
KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"
KUBE_API_ARGS=""
vi /etc/kubernetes/config
KUBE_LOGTOSTDERR="--logtostderr=true"
KUBE_LOG_LEVEL="--v=0"
KUBE_ALLOW_PRIV="--allow-privileged=false"
KUBE_MASTER="--master=http://172.16.159.130:8080"
3.3.3 配置kube-controller-manager
1 配置启动文件
vi /usr/lib/systemd/system/kube-controller-manager.service
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/controller-manager
ExecStart=/usr/bin/kube-controller-manager \
$KUBE_LOGTOSTDERR \
$KUBE_LOG_LEVEL \
$KUBE_MASTER \
$KUBE_CONTROLLER_MANAGER_ARGS
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
2 配置参数
vi /etc/kubernetes/controller-manager
KUBE_CONTROLLER_MANAGER_ARGS=" "
3.3.4 配置kube-scheduler
1 配置启动文件
vi /usr/lib/systemd/system/kube-scheduler.service
[Unit]
Description=Kubernetes Scheduler Plugin
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/scheduler
ExecStart=/usr/bin/kube-scheduler \
$KUBE_LOGTOSTDERR \
$KUBE_LOG_LEVEL \
$KUBE_MASTER \
$KUBE_SCHEDULER_ARGS
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
2 配置参数
mkdir -p /Data/logs/kubernetes
vi /etc/kubernetes/scheduler
KUBE_SCHEDULER_ARGS="--logtostderr=true --log-dir=/Data/logs/kubernetes --v=2"
3.3.5 将各组件加入开机自启
# systemctl daemon-reload
# systemctl enable kube-apiserver.service
# systemctl start kube-apiserver.service
# systemctl enable kube-controller-manager.service
# systemctl start kube-controller-manager.service
# systemctl enable kube-scheduler.service
# systemctl start kube-scheduler.service
3.3.6 验证 master 节点功能
# kubectl get componentstatuses
NAME STATUS MESSAGE ERROR
controller-manager Healthy ok
scheduler Healthy ok
etcd-0 Healthy {"health": "true"}
3.4 安装和配置 kubernetes(node节点部署)
k8s-node1,k8s-node2安装kubernetes。
3.4.1 安装kubernetes 二进制文件
# mkdir -p /Data/apps/k8s
# cd /Data/apps/k8s
# wget https://dl.k8s.io/v1.11.1/kubernetes-server-linux-amd64.tar.gz
# tar -xzvf kubernetes-server-linux-amd64.tar.gz
# cd kubernetes/server/bin/
# cp kube-proxy kubelet /usr/bin/
3.4.2 配置kubelet
1 配置kubelet启动项
vi /usr/lib/systemd/system/kubelet.service
[Unit]
Description=Kubernetes Kubelet
After=docker.service
Requires=docker.service
[Service]
EnvironmentFile=-/etc/kubernetes/kubelet
ExecStart=/usr/bin/kubelet \
${KUBE_LOGTOSTDERR} \
${KUBE_LOG_LEVEL} \
${NODE_ADDRESS} \
${NODE_PORT} \
${NODE_HOSTNAME} \
${KUBELET_KUBECONFIG} \
${KUBE_ALLOW_PRIV} \
${KUBELET_DNS_IP} \
${KUBELET_DNS_DOMAIN} \
${KUBELET_SWAP}
Restart=on-failure
KillMode=process
[Install]
WantedBy=multi-user.target
2 配置kubelet参数
mkdir -p /etc/kubernetes/
vi /etc/kubernetes/kubelet
启用日志标准错误
KUBE_LOGTOSTDERR="--logtostderr=true"
# 日志级别
KUBE_LOG_LEVEL="--v=0"
# Kubelet服务IP地址
NODE_ADDRESS="--address=172.16.159.131"
# Kubelet服务端口
NODE_PORT="--port=10250"
# 自定义节点名称
NODE_HOSTNAME="--hostname-override=172.16.159.131"
# kubeconfig路径,指定连接API服务器
KUBELET_KUBECONFIG="--kubeconfig=/etc/kubernetes/kubelet.kubeconfig"
# 允许容器请求特权模式,默认false
KUBE_ALLOW_PRIV="--allow-privileged=false"
#KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"
KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0"
# DNS信息
KUBELET_DNS_IP="--cluster-dns=10.254.0.2"
KUBELET_DNS_DOMAIN="--cluster-domain=cluster.local"
# 禁用使用Swap
KUBELET_SWAP="--fail-swap-on=false"
vi /etc/kubernetes/kubelet.kubeconfig
apiVersion: v1
kind: Config
clusters:
- cluster:
server: http://172.16.159.130:8080
name: local
contexts:
- context:
cluster: local
name: local
current-context: local
vi /etc/kubernetes/config
KUBE_LOGTOSTDERR="--logtostderr=true"
KUBE_LOG_LEVEL="--v=0"
KUBE_ALLOW_PRIV="--allow-privileged=false"
KUBE_MASTER="--master=http://172.16.159.130:8080"
3 启动 kubelet
# systemctl daemon-reload
# systemctl enable kubelet.service
# systemctl start kubelet.service
# systemctl status kubelet.service
3.4.3 配置kube-proxy
1 配置启动文件
vi /usr/lib/systemd/system/kube-proxy.service
[Unit]
Description=Kubernetes Kube-Proxy Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target
[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/proxy
ExecStart=/usr/bin/kube-proxy \
$KUBE_LOGTOSTDERR \
$KUBE_LOG_LEVEL \
$KUBE_MASTER \
$KUBE_PROXY_ARGS
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
2 配置参数
vi /etc/kubernetes/proxy
KUBE_PROXY_ARGS=""
3 启动 kube-proxy
# systemctl daemon-reload
# systemctl enable kube-proxy
# systemctl start kube-proxy
# systemctl status kube-proxy
3.4.4 检查节点状态
进入到k8s-master1机器
[root@k8s-master1 k8s]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
172.16.159.131 Ready <none> 17m v1.12.1
3.5 安装和配置 flanneld(master和node节点部署)
k8s-master1,k8s-node1,k8s-node2都得安装flanneld。
3.5.1 下载安装flanneld
# mkdir -p /Data/apps/flannel
# cd /Data/apps/flannel
# wget https://github.com/coreos/flannel/releases/download/v0.7.1/flannel-v0.7.1-linux-amd64.tar.gz
# tar -xzvf flannel-v0.7.1-linux-amd64.tar.gz
# cp flanneld mk-docker-opts.sh /usr/bin
3.5.2 配置flanneld
1 创建启动文件
vi /usr/lib/systemd/system/flanneld.service
[Unit]
Description=Flanneld overlay address etcd agent
After=network.target
After=network-online.target
Wants=network-online.target
After=etcd.service
Before=docker.service
[Service]
Type=notify
EnvironmentFile=/etc/sysconfig/flanneld
EnvironmentFile=-/etc/sysconfig/docker-network
ExecStart=/usr/bin/flanneld-start $FLANNEL_OPTIONS
ExecStartPost=/usr/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/docker
Restart=on-failure
[Install]
WantedBy=multi-user.target
WantedBy=docker.service
2 配置flanneld配置文件
mkdir -p /Data/apps/k8s/network
vi /etc/sysconfig/flanneld
# Flanneld configuration options
# etcd url location. Point this to the server where etcd runs
FLANNEL_ETCD_ENDPOINTS="http://172.16.159.130:2379"
# etcd config key. This is the configuration key that flannel queries
# For address range assignment
FLANNEL_ETCD_PREFIX="/Data/apps/k8s/network"
# Any additional options that you want to pass
#FLANNEL_OPTIONS=""
3 配置docker-network
vi /etc/sysconfig/docker-network
DOCKER_NETWORK_OPTIONS=
4 flanneld-star
vi /usr/bin/flanneld-start
#!/bin/sh
exec /usr/bin/flanneld \
-etcd-endpoints=${FLANNEL_ETCD_ENDPOINTS:-${FLANNEL_ETCD}} \
-etcd-prefix=${FLANNEL_ETCD_PREFIX:-${FLANNEL_ETCD_KEY}} \
"$@"
赋执行权限
chmod +x /usr/bin/flanneld-start
3.5.3 配置etcd中关于flannel的key(只用在k8s-master1上执行)
[root@k8s-master1 flannel]# etcdctl set /Data/apps/k8s/network/config '{"Network": "172.20.0.0/16"}'
{"Network": "172.20.0.0/16"}
[root@k8s-master1 flannel]# etcdctl get /Data/apps/k8s/network/config
{"Network": "172.20.0.0/16"}
[root@k8s-master1 flannel]#
3.5.4 启动flanneld
启动Flannel之后,需要依次重启docker、kubernete。
1 在master执行:
[root@k8s-master1 flannel]# systemctl daemon-reload
[root@k8s-master1 flannel]# systemctl enable flanneld.service
[root@k8s-master1 flannel]# systemctl start flanneld.service
[root@k8s-master1 flannel]# service docker restart
[root@k8s-master1 flannel]# systemctl restart kube-apiserver.service
[root@k8s-master1 flannel]# systemctl restart kube-controller-manager.service
[root@k8s-master1 flannel]# systemctl restart kube-scheduler.service
2 在node执行:
[root@k8s-master ~]# systemctl daemon-reload
[root@k8s-node-1 ~]# systemctl enable flanneld.service
[root@k8s-node-1 ~]# systemctl start flanneld.service
[root@k8s-node-1 ~]# service docker restart
[root@k8s-node-1 ~]# systemctl restart kubelet.service
[root@k8s-node-1 ~]# systemctl restart kube-proxy.service
4 参考文档
1 entOS7安装kubernetes1.11.2:
https://blog.csdn.net/zzq900503/article/details/81710319
2 二进制离线安装
https://www.jianshu.com/p/dadafc62ed24
http://blog.51cto.com/12480612/2287449?source=dra
https://blog.csdn.net/ljx1528/article/details/81545187
https://blog.csdn.net/a735131232/article/details/83352517 (有软件包)
http://blog.51cto.com/13120271/2115310
3 工作原理:
https://baijiahao.baidu.com/s?id=1602795888204860650&wfr=spider&for=pc
技术交流
CleverCode是一名架构师,技术交流,咨询问题,请加CleverCode创建的qq群(架构师俱乐部):517133582。加群和腾讯,阿里,百度,新浪等公司的架构师交流。【架构师俱乐部】宗旨:帮助你成长为架构师!