前期准备
配置hosts
192.168.245.227 iot-node1 \# master节点(etcd,kubernetes-master) 192.168.245.228 iot-node2 \# slave节点(etcd,kubernetes-node,docker) 192.168.245.229 iot-node3 \# slave节点(etcd,kubernetes-node,docker)
其他
同CentOS-7安装mesos+marathon+zk前期准备
添加安装源
添加etcd安装源
sudo rpm -Uvh
https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
sudo rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
添加kubernetes安装源
sudo tee /etc/yum.repos.d/kubernetes.repo <<-‘EOF’
[kubernetesrepo]
name=Kubernetes Repository
baseurl=http://cbs.centos.org/repos/virt7-kubernetes-110-release/x86_64/os
enabled=1
gpgcheck=1
gpgkey=https://yum.dockerproject.org/gpg
EOF
在线安装etcd
安装etcd
sudo yum install -y etcd
配置etcd
sudo vi /etc/etcd/etcd.conf
1)节点iot-node1中/etc/etcd/etcd.conf内容:
#[Member]
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_LISTEN_PEER_URLS=“http://192.168.245.227:2380”
ETCD_LISTEN_CLIENT_URLS=“http://192.168.245.227:2379,http://127.0.0.1:2379”
ETCD_NAME=“etcd1”
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS=“http://192.168.245.227:2380”
ETCD_ADVERTISE_CLIENT_URLS=“http://192.168.245.227:2379”
ETCD_INITIAL_CLUSTER=“etcd1=http://192.168.245.227:2380,etcd2=http://192.168.245.228:2380, etcd3=http://192.168.245.229:2380”
ETCD_INITIAL_CLUSTER_TOKEN=“etcd-cluster”
ETCD_INITIAL_CLUSTER_STATE=“new”
2)节点iot-node2中/etc/etcd/etcd.conf内容:
#[Member]
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_LISTEN_PEER_URLS=“http://192.168.245.228:2380”
ETCD_LISTEN_CLIENT_URLS="http://192.168.245.228:2379,http://127.0.0.1:2379 "
ETCD_NAME=“etcd1”
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS=“http://192.168.245.228:2380”
ETCD_ADVERTISE_CLIENT_URLS=“http://192.168.245.228:2379”
ETCD_INITIAL_CLUSTER=“etcd1=http://192.168.245.227:2380,etcd2=http://192.168.245.228:2380,
etcd3=http://192.168.245.229:2380”
ETCD_INITIAL_CLUSTER_TOKEN=“etcd-cluster”
ETCD_INITIAL_CLUSTER_STATE=“new”
3)节点iot-node3中/etc/etcd/etcd.conf内容:
#[Member]
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_LISTEN_PEER_URLS=“http://192.168.245.229:2380”
ETCD_LISTEN_CLIENT_URLS=“http://192.168.245.229:2379,http://127.0.0.1:2379”
ETCD_NAME=“etcd1”
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS=“http://192.168.245.229:2380”
ETCD_ADVERTISE_CLIENT_URLS=“http://192.168.245.229:2379”
ETCD_INITIAL_CLUSTER=“etcd1=http://192.168.245.227:2380,etcd2=http://192.168.245.228:2380,
etcd3=http://192.168.245.229:2380”
ETCD_INITIAL_CLUSTER_TOKEN=“etcd-cluster”
ETCD_INITIAL_CLUSTER_STATE=“new”
顺序启动3个节点的etcd,iot-node1、iot-node2两个节点启动etcd会报超
时,等所有节点都启动后就没问题了。
针对几个URLS做下简单的解释:
[member]
ETCD_NAME :ETCD的节点名
ETCD_DATA_DIR:ETCD的数据存储目录
ETCD_SNAPSHOT_COUNTER:多少次的事务提交将触发一次快照
ETCD_HEARTBEAT_INTERVAL:ETCD节点之间心跳传输的间隔,单位毫秒
ETCD_ELECTION_TIMEOUT:该节点参与选举的最大超时时间,单位毫秒
ETCD_LISTEN_PEER_URLS:该节点与其他节点通信时所监听的地址列表,多个地址使用逗号隔开,其格式可以划分为scheme://IP:PORT,这里的scheme可以是http、https
ETCD_LISTEN_CLIENT_URLS:该节点与客户端通信时监听的地址列表
[cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS:该成员节点在整个集群中的通信地址列表,这个地址用来传输集群数据的地址。因此这个地址必须是可以连接集群中所有的成员的。
ETCD_INITIAL_CLUSTER:配置集群内部所有成员地址,其格式为:ETCD_NAME=ETCD_INITIAL_ADVERTISE_PEER_URLS,如果有多个使用逗号隔开
ETCD_ADVERTISE_CLIENT_URLS:广播给集群中其他成员自己的客户端地址列表
启动/停止/查看etcd
sudo systemctl start etcd
sudo systemctl stop etcd
sudo systemctl status etcd
设置开机启动
sudo systemctl enable etcd
查看etcd启动日志
sudo journalctl -xef
查看etcd集群节点
etcdctl member list
查看etcd集群状态
etcdctl cluster-health
在线安装kubernetes
安装kubernetes
1)master节点安装
sudo yum install -y kubernetes-master
2)slave节点安装
sudo yum install -y kubernetes-node
配置kubernetes
1)所有节点配置config
sudo vi /etc/kubernetes/config
/etc/kubernetes/config内容:
###
# kubernetes system config
#
# The following values are used to configure various aspects of all
# kubernetes services, including
#
# kube-apiserver.service
# kube-controller-manager.service
# kube-scheduler.service
# kubelet.service
# kube-proxy.service
# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="–logtostderr=true"
# journal message level, 0 is debug
KUBE_LOG_LEVEL="–v=0"
# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="–allow-privileged=false"
# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="–master=http://192.168.245.227:8080"
2)master节点配置apiserver
sudo vi /etc/kubernetes/apiserver
/etc/kubernetes/apiserver内容:
# kubernetes system config
#
# The following values are used to configure the kube-apiserver
#
# The address on the local server to listen to.
KUBE_API_ADDRESS="–address=0.0.0.0"
# The port on the local server to listen on.
KUBE_API_PORT="–port=8080"
# Port minions listen on
KUBELET_PORT="–kubelet-port=10250"
# Comma separated list of nodes in the etcd cluster
KUBE_ETCD_SERVERS="–etcd-servers=http:// 192.168.245.227:2379, http://
192.168.245.228:2379,http:// 192.168.245.229:2379 "
# Address range to use for services
KUBE_SERVICE_ADDRESSES="–service-cluster-ip-range=172.16.0.0/24"
# default admission control policies
KUBE_ADMISSION_CONTROL="–admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,ResourceQuota"
# Add your own!
KUBE_API_ARGS="–service-node-port-range=1-65535 --insecure-bind-address=0.0.0.0"
2)master节点配置controller-manager
sudo vi /etc/kubernetes/controller-manager
/etc/kubernetes/controller-manager内容:
###
# The following values are used to configure the kubernetes controller-manager
# defaults from config and apiserver should be adequate
# Add your own!
KUBE_CONTROLLER_MANAGER_ARGS=""
3)master节点配置scheduler
sudo vi /etc/kubernetes/scheduler
/etc/kubernetes/scheduler:
###
# The following values are used to configure the kubernetes scheduler
# defaults from config and apiserver should be adequate
# Add your own!
KUBE_SCHEDULER_ARGS=""
4)slave节点配置kubelet
sudo vi /etc/kubernetes/kubelet
/etc/kubernetes/kubelet内容:
###
# kubernetes kubelet (minion) config
# The address for the info server to serve on (set to 0.0.0.0 or “” for all
interfaces)
KUBELET_ADDRESS="–address=0.0.0.0"
# The port for the info server to serve on
# KUBELET_PORT="–port=10250"
# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="–hostname-override=192.168.245.228"
# location of the api-server
# KUBELET_API_SERVER="–api-servers=http://192.168.245.227:8080"
# pod infrastructure container
KUBELET_POD_INFRA_CONTAINER="–pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"
# Add your own!
KUBELET_ARGS="–kubeconfig=/etc/kubernetes/kubeconfig
–runtime-cgroups=/systemd/system.slice
–kubelet-cgroups=/systemd/system.slice --cluster-dns=172.16.0.0
–cgroup-driver=systemd --enable-server=true
–enable-debugging-handlers=true"
注意:
不同slave需修改KUBELET_HOSTNAME
这里需要说明的是kubernetesV1.9
开始已经不支持–api-servers参数,需要指定-- kubeconfig=/etc/kubernetes/kubeconfig配置文件来指明master的地址,否则master主机执行kubectl get nodes,会提示找不到节点
5)slave节点配置kubeconfig
sudo vi /etc/kubernetes/kubeconfig
/etc/kubernetes/kubeconfig内容:
apiVersion: v1
kind: Config
users:
- name: kubelet
clusters:
- name: kubernetes
cluster:
server: http://192.168.245.227:8080
contexts:
- context:
cluster: kubernetes
user: kubelet
name: service-account-context
current-context: service-account-context
6)slave节点配置proxy
sudo vi /etc/kubernetes/proxy
/etc/kubernetes/proxy内容:
# kubernetes proxy config
# default config should be adequate
# Add your own!
KUBE_PROXY_ARGS="–bind-address=192.168.245.228
–hostname-override=192.168.245.228"
注意:不同slave需修改KUBE_PROXY_ARGS中的ip
7)修改docker cgroupdriver
sudo vi /etc/docker/daemon.json
/etc/docker/daemon.json内容:
{
“storage-driver”: “devicemapper”,
“storage-opts”: [
“dm.thinpooldev=/dev/mapper/docker-thinpool”,
“dm.use_deferred_removal=true”,
“dm.use_deferred_deletion=true”]
, “exec-opts”:[ “native.cgroupdriver=systemd” ]
}
master节点启动服务
sudo systemctl daemon-reload
sudo systemctl start kube-apiserver
sudo systemctl start kube-controller-manager
sudo systemctl start kube-scheduler
master节点开机启动
sudo systemctl enable kube-apiserver
sudo systemctl enable kube-controller-manager
sudo systemctl enable kube-scheduler
slave节点启动服务
sudo systemctl daemon-reload
sudo systemctl start kubelet
sudo systemctl start kube-proxy
slave节点开机启动
sudo systemctl enable kubelet
sudo systemctl enable kube-proxy
验证kubernetes集群
在master节点,执行
kubectl get nodes
在master节点,执行
kubectl get node --show-labels=true
在master节点,执行
kubectl get cs
离线安装etcd
下载地址
http://www.rpmfind.net/linux/centos/7.5.1804/extras/x86_64/Packages/
安装包
etcd-3.2.22-1.el7.x86_64.rpm
安装etcd
进入到etcd安装包和依赖包所在目录执行命令:
sudo rpm -ivh *.rpm
配置etcd
相关配置参考etcd在线安装
离线安装kubernetes
下载地址
http://cbs.centos.org/repos/virt7-kubernetes-110-release/x86_64/os/Packages/
安装包
kubernetes-master-1.10.3-0.el7.x86_64.rpm
kubernetes-client-1.10.3-0.el7.x86_64.rpm
kubernetes-node-1.10.3-0.el7.x86_64.rpm
依赖包
PyYAML-3.10-11.el7.x86_64.rpm
atomic-registries-1.22.1-22.git5a342e3.el7.x86_64.rpm
checkpolicy-2.5-6.el7.x86_64.rpm
conntrack-tools-1.4.4-3.el7_3.x86_64.rpm
audit-libs-python-2.8.1-3.el7_5.1.x86_64.rpm
container-storage-setup-0.11.0-2.git5eaf76c.el7.noarch.rpm
container-selinux-2.68-1.el7.noarch.rpm
docker-client-1.13.1-74.git6e3bb8e.el7.centos.x86_64.rpm
docker-common-1.13.1-74.git6e3bb8e.el7.centos.x86_64.rpm
libnetfilter_cthelper-1.0.0-9.el7.x86_64.rpm
libnetfilter_cttimeout-1.0.0-6.el7.x86_64.rpm
libnetfilter_queue-1.0.2-2.el7_2.x86_64.rpm
libseccomp-2.3.1-3.el7.x86_64.rpm
libcgroup-0.41-15.el7.x86_64.rpm
libsemanage-python-2.5-11.el7.x86_64.rpm
docker-1.13.1-74.git6e3bb8e.el7.centos.x86_64.rpm
libyaml-0.1.4-11.el7_0.x86_64.rpm
oci-register-machine-0-6.git2b44233.el7.x86_64.rpm
oci-umount-2.3.3-3.gite3c9055.el7.x86_64.rpm
oci-systemd-hook-0.1.17-2.git83283a0.el7.x86_64.rpm
python-IPy-0.75-6.el7.noarch.rpm
policycoreutils-python-2.5-22.el7.x86_64.rpm
python-backports-1.0-8.el7.x86_64.rpm
python-backports-ssl_match_hostname-3.5.0.1-1.el7.noarch.rpm
python-ipaddress-1.0.16-2.el7.noarch.rpm
python-setuptools-0.9.8-7.el7.noarch.rpm
skopeo-containers-0.1.31-1.dev.gitae64ff7.el7.centos.x86_64.rpm
setools-libs-3.3.8-2.el7.x86_64.rpm
python-pytoml-0.1.14-1.git7dea353.el7.noarch.rpm
socat-1.7.3.2-2.el7.x86_64.rpm
subscription-manager-rhsm-certificates-1.20.11-1.el7.centos.x86_64.rpm
yajl-2.0.4-4.el7.x86_64.rpm
安装kubernetes
进入到kubernetes安装包和依赖包所在目录执行命令:
sudo rpm -ivh *.rpm
配置kubernetes
相关配置参考kubernetes在线安装