CentOS-7安装kubernetes+etcd

前期准备

配置hosts

192.168.245.227 iot-node1 \# master节点(etcd,kubernetes-master)
192.168.245.228 iot-node2 \# slave节点(etcd,kubernetes-node,docker)
192.168.245.229 iot-node3 \# slave节点(etcd,kubernetes-node,docker)

其他

同CentOS-7安装mesos+marathon+zk前期准备

添加安装源

添加etcd安装源

sudo rpm -Uvh
https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

sudo rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6

添加kubernetes安装源

sudo tee /etc/yum.repos.d/kubernetes.repo <<-‘EOF’
[kubernetesrepo]
name=Kubernetes Repository
baseurl=http://cbs.centos.org/repos/virt7-kubernetes-110-release/x86_64/os
enabled=1
gpgcheck=1
gpgkey=https://yum.dockerproject.org/gpg
EOF

在线安装etcd

安装etcd

sudo yum install -y etcd

配置etcd

sudo vi /etc/etcd/etcd.conf

1）节点iot-node1中/etc/etcd/etcd.conf内容：

#[Member]
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_LISTEN_PEER_URLS=“http://192.168.245.227:2380”
ETCD_LISTEN_CLIENT_URLS=“http://192.168.245.227:2379,http://127.0.0.1:2379”
ETCD_NAME=“etcd1”
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS=“http://192.168.245.227:2380”
ETCD_ADVERTISE_CLIENT_URLS=“http://192.168.245.227:2379”
ETCD_INITIAL_CLUSTER=“etcd1=http://192.168.245.227:2380,etcd2=http://192.168.245.228:2380, etcd3=http://192.168.245.229:2380”
ETCD_INITIAL_CLUSTER_TOKEN=“etcd-cluster”
ETCD_INITIAL_CLUSTER_STATE=“new”

2）节点iot-node2中/etc/etcd/etcd.conf内容：

#[Member]
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_LISTEN_PEER_URLS=“http://192.168.245.228:2380”
ETCD_LISTEN_CLIENT_URLS="http://192.168.245.228:2379,http://127.0.0.1:2379 "
ETCD_NAME=“etcd1”
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS=“http://192.168.245.228:2380”
ETCD_ADVERTISE_CLIENT_URLS=“http://192.168.245.228:2379”
ETCD_INITIAL_CLUSTER=“etcd1=http://192.168.245.227:2380,etcd2=http://192.168.245.228:2380,
etcd3=http://192.168.245.229:2380”
ETCD_INITIAL_CLUSTER_TOKEN=“etcd-cluster”
ETCD_INITIAL_CLUSTER_STATE=“new”

3）节点iot-node3中/etc/etcd/etcd.conf内容：

#[Member]
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_LISTEN_PEER_URLS=“http://192.168.245.229:2380”
ETCD_LISTEN_CLIENT_URLS=“http://192.168.245.229:2379,http://127.0.0.1:2379”
ETCD_NAME=“etcd1”
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS=“http://192.168.245.229:2380”
ETCD_ADVERTISE_CLIENT_URLS=“http://192.168.245.229:2379”
ETCD_INITIAL_CLUSTER=“etcd1=http://192.168.245.227:2380,etcd2=http://192.168.245.228:2380,
etcd3=http://192.168.245.229:2380”
ETCD_INITIAL_CLUSTER_TOKEN=“etcd-cluster”
ETCD_INITIAL_CLUSTER_STATE=“new”

顺序启动3个节点的etcd，iot-node1、iot-node2两个节点启动etcd会报超
时，等所有节点都启动后就没问题了。

针对几个URLS做下简单的解释：

[member]
ETCD_NAME ：ETCD的节点名
ETCD_DATA_DIR：ETCD的数据存储目录
ETCD_SNAPSHOT_COUNTER：多少次的事务提交将触发一次快照
ETCD_HEARTBEAT_INTERVAL：ETCD节点之间心跳传输的间隔，单位毫秒
ETCD_ELECTION_TIMEOUT：该节点参与选举的最大超时时间，单位毫秒
ETCD_LISTEN_PEER_URLS：该节点与其他节点通信时所监听的地址列表，多个地址使用逗号隔开，其格式可以划分为scheme://IP:PORT，这里的scheme可以是http、https
ETCD_LISTEN_CLIENT_URLS：该节点与客户端通信时监听的地址列表
[cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS：该成员节点在整个集群中的通信地址列表，这个地址用来传输集群数据的地址。因此这个地址必须是可以连接集群中所有的成员的。
ETCD_INITIAL_CLUSTER：配置集群内部所有成员地址，其格式为：ETCD_NAME=ETCD_INITIAL_ADVERTISE_PEER_URLS，如果有多个使用逗号隔开
ETCD_ADVERTISE_CLIENT_URLS：广播给集群中其他成员自己的客户端地址列表

启动/停止/查看etcd

sudo systemctl start etcd

sudo systemctl stop etcd

sudo systemctl status etcd

设置开机启动

sudo systemctl enable etcd

查看etcd启动日志

sudo journalctl -xef

查看etcd集群节点

etcdctl member list

查看etcd集群状态

etcdctl cluster-health

在线安装kubernetes

安装kubernetes

1）master节点安装

sudo yum install -y kubernetes-master

2）slave节点安装

sudo yum install -y kubernetes-node

配置kubernetes

1）所有节点配置config

sudo vi /etc/kubernetes/config

/etc/kubernetes/config内容：

###
# kubernetes system config
#
# The following values are used to configure various aspects of all
# kubernetes services, including
#
# kube-apiserver.service
# kube-controller-manager.service
# kube-scheduler.service
# kubelet.service
# kube-proxy.service
# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="–logtostderr=true"
# journal message level, 0 is debug
KUBE_LOG_LEVEL="–v=0"
# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="–allow-privileged=false"
# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="–master=http://192.168.245.227:8080"

2）master节点配置apiserver

sudo vi /etc/kubernetes/apiserver

/etc/kubernetes/apiserver内容：

# kubernetes system config
#
# The following values are used to configure the kube-apiserver
#
# The address on the local server to listen to.
KUBE_API_ADDRESS="–address=0.0.0.0"
# The port on the local server to listen on.
KUBE_API_PORT="–port=8080"
# Port minions listen on
KUBELET_PORT="–kubelet-port=10250"
# Comma separated list of nodes in the etcd cluster
KUBE_ETCD_SERVERS="–etcd-servers=http:// 192.168.245.227:2379, http://
192.168.245.228:2379,http:// 192.168.245.229:2379 "
# Address range to use for services
KUBE_SERVICE_ADDRESSES="–service-cluster-ip-range=172.16.0.0/24"
# default admission control policies
KUBE_ADMISSION_CONTROL="–admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,ResourceQuota"
# Add your own!
KUBE_API_ARGS="–service-node-port-range=1-65535 --insecure-bind-address=0.0.0.0"

2）master节点配置controller-manager

sudo vi /etc/kubernetes/controller-manager

/etc/kubernetes/controller-manager内容：

###
# The following values are used to configure the kubernetes controller-manager
# defaults from config and apiserver should be adequate
# Add your own!
KUBE_CONTROLLER_MANAGER_ARGS=""

3）master节点配置scheduler

sudo vi /etc/kubernetes/scheduler

/etc/kubernetes/scheduler：

###
# The following values are used to configure the kubernetes scheduler
# defaults from config and apiserver should be adequate
# Add your own!
KUBE_SCHEDULER_ARGS=""

4）slave节点配置kubelet

sudo vi /etc/kubernetes/kubelet

/etc/kubernetes/kubelet内容：

###
# kubernetes kubelet (minion) config
# The address for the info server to serve on (set to 0.0.0.0 or “” for all
interfaces)
KUBELET_ADDRESS="–address=0.0.0.0"
# The port for the info server to serve on
# KUBELET_PORT="–port=10250"
# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="–hostname-override=192.168.245.228"
# location of the api-server
# KUBELET_API_SERVER="–api-servers=http://192.168.245.227:8080"
# pod infrastructure container
KUBELET_POD_INFRA_CONTAINER="–pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"
# Add your own!
KUBELET_ARGS="–kubeconfig=/etc/kubernetes/kubeconfig
–runtime-cgroups=/systemd/system.slice
–kubelet-cgroups=/systemd/system.slice --cluster-dns=172.16.0.0
–cgroup-driver=systemd --enable-server=true
–enable-debugging-handlers=true"

注意：

不同slave需修改KUBELET_HOSTNAME

这里需要说明的是kubernetesV1.9
开始已经不支持–api-servers参数，需要指定-- kubeconfig=/etc/kubernetes/kubeconfig配置文件来指明master的地址，否则master主机执行kubectl get nodes，会提示找不到节点

5）slave节点配置kubeconfig

sudo vi /etc/kubernetes/kubeconfig

/etc/kubernetes/kubeconfig内容：

apiVersion: v1
kind: Config
users:
- name: kubelet
clusters:
- name: kubernetes
cluster:
server: http://192.168.245.227:8080
contexts:
- context:
cluster: kubernetes
user: kubelet
name: service-account-context
current-context: service-account-context

6）slave节点配置proxy

sudo vi /etc/kubernetes/proxy

/etc/kubernetes/proxy内容：

# kubernetes proxy config
# default config should be adequate
# Add your own!
KUBE_PROXY_ARGS="–bind-address=192.168.245.228
–hostname-override=192.168.245.228"

注意：不同slave需修改KUBE_PROXY_ARGS中的ip

7）修改docker cgroupdriver

sudo vi /etc/docker/daemon.json

/etc/docker/daemon.json内容：

{
“storage-driver”: “devicemapper”,
“storage-opts”: [
“dm.thinpooldev=/dev/mapper/docker-thinpool”,
“dm.use_deferred_removal=true”,
“dm.use_deferred_deletion=true”]
, “exec-opts”:[ “native.cgroupdriver=systemd” ]
}

master节点启动服务

sudo systemctl daemon-reload

sudo systemctl start kube-apiserver

sudo systemctl start kube-controller-manager

sudo systemctl start kube-scheduler

master节点开机启动

sudo systemctl enable kube-apiserver

sudo systemctl enable kube-controller-manager

sudo systemctl enable kube-scheduler

slave节点启动服务

sudo systemctl daemon-reload

sudo systemctl start kubelet

sudo systemctl start kube-proxy

slave节点开机启动

sudo systemctl enable kubelet

sudo systemctl enable kube-proxy

验证kubernetes集群

在master节点，执行

kubectl get nodes

在master节点，执行

kubectl get node --show-labels=true

在master节点，执行

kubectl get cs

离线安装etcd

下载地址

http://www.rpmfind.net/linux/centos/7.5.1804/extras/x86_64/Packages/

安装包

etcd-3.2.22-1.el7.x86_64.rpm

安装etcd

进入到etcd安装包和依赖包所在目录执行命令：

sudo rpm -ivh *.rpm

配置etcd

相关配置参考etcd在线安装

离线安装kubernetes

下载地址

http://cbs.centos.org/repos/virt7-kubernetes-110-release/x86_64/os/Packages/

安装包

kubernetes-master-1.10.3-0.el7.x86_64.rpm
kubernetes-client-1.10.3-0.el7.x86_64.rpm
kubernetes-node-1.10.3-0.el7.x86_64.rpm

依赖包

PyYAML-3.10-11.el7.x86_64.rpm
atomic-registries-1.22.1-22.git5a342e3.el7.x86_64.rpm
checkpolicy-2.5-6.el7.x86_64.rpm
conntrack-tools-1.4.4-3.el7_3.x86_64.rpm
audit-libs-python-2.8.1-3.el7_5.1.x86_64.rpm
container-storage-setup-0.11.0-2.git5eaf76c.el7.noarch.rpm
container-selinux-2.68-1.el7.noarch.rpm
docker-client-1.13.1-74.git6e3bb8e.el7.centos.x86_64.rpm
docker-common-1.13.1-74.git6e3bb8e.el7.centos.x86_64.rpm
libnetfilter_cthelper-1.0.0-9.el7.x86_64.rpm
libnetfilter_cttimeout-1.0.0-6.el7.x86_64.rpm
libnetfilter_queue-1.0.2-2.el7_2.x86_64.rpm
libseccomp-2.3.1-3.el7.x86_64.rpm
libcgroup-0.41-15.el7.x86_64.rpm
libsemanage-python-2.5-11.el7.x86_64.rpm
docker-1.13.1-74.git6e3bb8e.el7.centos.x86_64.rpm
libyaml-0.1.4-11.el7_0.x86_64.rpm
oci-register-machine-0-6.git2b44233.el7.x86_64.rpm
oci-umount-2.3.3-3.gite3c9055.el7.x86_64.rpm
oci-systemd-hook-0.1.17-2.git83283a0.el7.x86_64.rpm
python-IPy-0.75-6.el7.noarch.rpm
policycoreutils-python-2.5-22.el7.x86_64.rpm
python-backports-1.0-8.el7.x86_64.rpm
python-backports-ssl_match_hostname-3.5.0.1-1.el7.noarch.rpm
python-ipaddress-1.0.16-2.el7.noarch.rpm
python-setuptools-0.9.8-7.el7.noarch.rpm
skopeo-containers-0.1.31-1.dev.gitae64ff7.el7.centos.x86_64.rpm
setools-libs-3.3.8-2.el7.x86_64.rpm
python-pytoml-0.1.14-1.git7dea353.el7.noarch.rpm
socat-1.7.3.2-2.el7.x86_64.rpm
subscription-manager-rhsm-certificates-1.20.11-1.el7.centos.x86_64.rpm
yajl-2.0.4-4.el7.x86_64.rpm

安装kubernetes

进入到kubernetes安装包和依赖包所在目录执行命令：

sudo rpm -ivh *.rpm

配置kubernetes

相关配置参考kubernetes在线安装