结构:
- Pods
在Kubernetes系统中,调度的最小颗粒不是单纯的容器,而是抽象成一个Pod,Pod是一个可以被创建、销毁、调度、管理的最小的部署单元。比如一个或一组容器。 - Replication Controllers
Replication Controller是Kubernetes系统中最有用的功能,实现复制多个Pod副本,往往一个应用需要多个Pod来支撑,并且可以保证其复制的副本数,即使副本所调度分配的主宿机出现异常,通过Replication Controller可以保证在其它主宿机启用同等数量的Pod。Replication Controller可以通过repcon模板来创建多个Pod副本,同样也可以直接复制已存在Pod,需要通过Label selector来关联。 - Services
Services是Kubernetes最外围的单元,通过虚拟一个访问IP及服务端口,可以访问我们定义好的Pod资源,目前的版本是通过iptables的nat转发来实现,转发的目标端口为Kube_proxy生成的随机端口,目前只提供GOOGLE云上的访问调度,如GCE。如果与我们自建的平台进行整合?请关注下篇《kubernetes与HECD架构的整合》文章。 - Labels
Labels是用于区分Pod、Service、Replication Controller的key/value键值对,仅使用在Pod、Service、 Replication Controller之间的关系识别,但对这些单元本身进行操作时得使用name标签。 - Proxy
Proxy不但解决了同一主宿机相同服务端口冲突的问题,还提供了Service转发服务端口对外提供服务的能力,Proxy后端使用了随机、轮循负载均衡算法。 - Deployment
Kubernetes Deployment提供了官方的用于更新Pod和Replica Set(下一代的Replication Controller)的方法Kubernetes Deployment提供了官方的用于更新Pod和Replica Set(下一代的Replication Controller)的方法,您可以在Deployment对象中只描述您所期望的理想状态(预期的运行状态),Deployment控制器为您将现在的实际状态转换成您期望的状态,例如,您想将所有的webapp:v1.0.9升级成webapp:v1.1.0,您只需创建一个Deployment,Kubernetes会按照Deployment自动进行升级。现在,您可以通过Deployment来创建新的资源(pod,rs,rc),替换已经存在的资源等。
Deployment集成了上线部署、滚动升级、创建副本、暂停上线任务,恢复上线任务,回滚到以前某一版本(成功/稳定)的Deployment等功能,在某种程度上,Deployment可以帮我们实现无人值守的上线,大大降低我们的上线过程的复杂沟通、操作风险。 - etcd是Kubernetes提供默认的存储系统,保存所有集群数据,使用时需要为etcd数据提供备份计划。
机器准备
节点及功能 | 主机名 | IP |
---|---|---|
Master、etcd、registry | master | 192.168.20.22 |
Node1 | node01 | 192.168.20.86 |
Node2 | node02 | 192.168.20.88 |
参考链接:
使用 yum 安装 Kubernetes 集群:https://blog.51cto.com/wzlinux/2321767
使用 二进制安装 Kubernetes 集群:https://blog.51cto.com/wzlinux/2322345
1.修改hostname并关闭防火墙
192.168.20.22 master.wzlinux.com master
192.168.20.86 node01.wzlinux.com node01
192.168.20.88 node02.wzlinux.com node02
systemctl disable firewalld.service
systemctl stop firewalld.service
关闭SELinux。
sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux/config
setenforce 0
关闭swap。
swapoff -a
sed -i 's/.*swap.*/#&/' /etc/fstab
2.安装master
yum install kubernetes-master etcd -y
配置etcd
cat /etc/etcd/etcd.conf
#[Member]
#ETCD_CORS=""
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
#ETCD_WAL_DIR=""
#ETCD_LISTEN_PEER_URLS="http://localhost:2380"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
ETCD_NAME="default"
#ETCD_SNAPSHOT_COUNT="100000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
#ETCD_QUOTA_BACKEND_BYTES="0"
#ETCD_MAX_REQUEST_BYTES="1572864"
#ETCD_GRPC_KEEPALIVE_MIN_TIME="5s"
#ETCD_GRPC_KEEPALIVE_INTERVAL="2h0m0s"
#ETCD_GRPC_KEEPALIVE_TIMEOUT="20s"
#
#[Clustering]
#ETCD_INITIAL_ADVERTISE_PEER_URLS="http://localhost:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://0.0.0.0:2379"
#ETCD_DISCOVERY=""
#ETCD_DISCOVERY_FALLBACK="proxy"
#ETCD_DISCOVERY_PROXY=""
#ETCD_DISCOVERY_SRV=""
#ETCD_INITIAL_CLUSTER="default=http://localhost:2380"
#ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
#ETCD_INITIAL_CLUSTER_STATE="new"
#ETCD_STRICT_RECONFIG_CHECK="true"
#ETCD_ENABLE_V2="true"
#
#[Proxy]
#ETCD_PROXY="off"
#ETCD_PROXY_FAILURE_WAIT="5000"
#ETCD_PROXY_REFRESH_INTERVAL="30000"
#ETCD_PROXY_DIAL_TIMEOUT="1000"
#ETCD_PROXY_WRITE_TIMEOUT="5000"
#ETCD_PROXY_READ_TIMEOUT="0"
#
#[Security]
#ETCD_CERT_FILE=""
#ETCD_KEY_FILE=""
#ETCD_CLIENT_CERT_AUTH="false"
#ETCD_TRUSTED_CA_FILE=""
#ETCD_AUTO_TLS="false"
#ETCD_PEER_CERT_FILE=""
#ETCD_PEER_KEY_FILE=""
#ETCD_PEER_CLIENT_CERT_AUTH="false"
#ETCD_PEER_TRUSTED_CA_FILE=""
#ETCD_PEER_AUTO_TLS="false"
#
#[Logging]
#ETCD_DEBUG="false"
#ETCD_LOG_PACKAGE_LEVELS=""
#ETCD_LOG_OUTPUT="default"
#
#[Unsafe]
#ETCD_FORCE_NEW_CLUSTER="false"
#
#[Version]
#ETCD_VERSION="false"
#ETCD_AUTO_COMPACTION_RETENTION="0"
#
#[Profiling]
#ETCD_ENABLE_PPROF="false"
#ETCD_METRICS="basic"
#
#[Auth]
#ETCD_AUTH_TOKEN="simple"
启动服务
systemctl start etcd
systemctl enable etcd
查看启动状态
yum install net-tools
netstat -tlnp|grep etcd
cat /etc/kubernetes/apiserver
###
# kubernetes system config
#
# The following values are used to configure the kube-apiserver
#
# The address on the local server to listen to.
#KUBE_API_ADDRESS="--insecure-bind-address=127.0.0.1"
KUBE_API_ADDRESS="--address=0.0.0.0"
# The port on the local server to listen on.
KUBE_API_PORT="--port=8080"
# Port minions listen on
KUBELET_PORT="--kubelet-port=10250"
# Comma separated list of nodes in the etcd cluster
KUBE_ETCD_SERVERS="--etcd-servers=http://192.168.20.22:2379"
# Address range to use for services
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
# default admission control policies
#KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota"
KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,ResourceQuota"
# Add your own!
KUBE_API_ARGS=""
3.启动服务
systemctl start kube-apiserver
systemctl start kube-controller-manager
systemctl start kube-scheduler
systemctl enable kube-apiserver;systemctl enable kube-controller-manager;systemctl enable kube-scheduler
netstat -tlnp|grep kube-apiserver
netstat -tlnp|grep kube-scheduler
4.配置node节点
yum install kubernetes-node flannel docker -y
配置镜像加速
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
"registry-mirrors": ["https://2kmdn5ce.mirror.aliyuncs.com"]
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker
//node节点编辑配置文件
cat /etc/kubernetes/config
###
# kubernetes system config
#
# The following values are used to configure various aspects of all
# kubernetes services, including
#
# kube-apiserver.service
# kube-controller-manager.service
# kube-scheduler.service
# kubelet.service
# kube-proxy.service
# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=true"
# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"
# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow-privileged=false"
# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=http://192.168.20.22:8080"
//配置 kubelet 服务
vi /etc/kubernetes/kubelet
###
# kubernetes kubelet (minion) config
# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
KUBELET_ADDRESS="--address=0.0.0.0"
# The port for the info server to serve on
#KUBELET_PORT="--port=10250"
# You may leave this blank to use the actual hostname
#KUBELET_HOSTNAME="--hostname-override=192.168.20.86"
# location of the api-server
KUBELET_API_SERVER="--api-servers=http://192.168.20.22:8080"
# pod infrastructure container
#KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"
# Add your own!
KUBELET_ARGS="--cluster-dns=192.168.20.22 --cluster-domain=playcrab-inc.com"
//启动服务
systemctl start kubelet
systemctl start kube-proxy
systemctl enable kubelet
systemctl enable kube-proxy
5.配置 flannel 网络
vi /etc/sysconfig/flanneld
# Flanneld configuration options
# etcd url location. Point this to the server where etcd runs
FLANNEL_ETCD_ENDPOINTS="http://192.168.20.22:2379"
# etcd config key. This is the configuration key that flannel queries
# For address range assignment
FLANNEL_ETCD_PREFIX="/atomic.io/network"
# Any additional options that you want to pass
#FLANNEL_OPTIONS=""
//在master节点上为 falnnel 创建分配的网络。
etcdctl mk /atomic.io/network/config ‘{“Network”: “10.244.0.0/16”}’
//在各node节点上启动 flannel。
systemctl start flanneld
systemctl enable flanneld
6.启动 docker 服务===因为要让pod使用flannel创建的网络,所以docker的启动要在flannel后面。
systemctl restart docker
systemctl enable docker
7.启动验收
[root@master ~]# kubectl get nodes
NAME STATUS AGE
node01.wzlinux.com Ready 12m
node02.wzlinux.com Ready 10m
创建pod。
kubectl run nginx --image=nginx --replicas=3
发现容器一直在创建。
[root@master ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE
nginx-701339712-099j9 0/1 ContainerCreating 0 3m <none> node01.wzlinux.com
nginx-701339712-vw0zh 0/1 ContainerCreating 0 3m <none> node02.wzlinux.com
nginx-701339712-wqjq1 0/1 ContainerCreating 0 3m <none> node02.wzlinux.com
这种情况,我们可以使用kubectl describe pod POD_NAME
查看Pod因为什么原因一直卡主。
因为网络的原因,这个地方我们访问不到,不过我们可以间接的获取这个镜像,那我们通过阿里云上面的镜像来替代,在所以node节点执行下面的命令。
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.0
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.0 gcr.io/google_containers/pause-amd64:3.0
docker image rm registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.0
稍等片刻,我们在查看pod状态,发现已经OK,如果还有问题,可以继续进入容器查看日志。
[root@master ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE
nginx-701339712-099j9 1/1 Running 0 11m 10.244.32.2 node01.wzlinux.com
nginx-701339712-vw0zh 1/1 Running 0 11m 10.244.53.3 node02.wzlinux.com
nginx-701339712-wqjq1 1/1 Running 0 11m 10.244.53.2 node02.wzlinux.com
但是还有问题,这是因为iptables的FORWARD是默认禁止的,禁用了iptables filter表中FOWARD链,这样会引起Kubernetes集群中跨Node的Pod无法通信。
我们这里手动在所有node节点上面放行。
iptables -P FORWARD ACCEPT