docker集群管理工具kubernetes初步搭建

前言：Kubernetes 是Google开源的容器集群管理系统，它构建于docker技术之上，基于Docker构建一个容器的调度服务，提供资源调度、均衡容灾、服务注册、动态扩缩容等功能套件，本质上可看作是基于容器技术的mini-PaaS平台。本文介绍如何基于Centos7.2构建Kubernetes多节点平台，了解 Kubernetes 的基本概念和多节点部署架构将使安装和管理更容易。建议在继续前进之前，查阅一些Kubernetes 介绍文档,这里就不对k8s的相关概念做介绍了。

• 官方原文介绍
• Kubernetes系统架构简介

---

截止至2015年9月1日，CentOS 已经把 kubernetes 加入官方源，所以现在安装k8s的各组件只需yum一下

一、环境说明

1. 各组件版本

• Kubernetes-1.2.0
• docker-1.10.3
• flannel-0.5.3
• etcd-2.3.7
• go-1.6.3

2. kubernetes 集群架构图：

3. kubernetes 各角色：

• 1.1.1.100 master/etc/
• 1.1.1.101 minion(node)
• 1.1.1.102 minion(node)
  以下均用IP进行操作，但是建议部署的时候，建议用hosts或者DNS取代IP。

4. 各组件用途

kube maste：

• kube-apiserver
  k8s的管理接口

• kube-scheduer
  k8s调度器，容器的启动、迁移、扩容缩减时候，选择哪个node，就看它了。

• kube-controller-manager
  k8s对node的控制行为，比如怎么去调用node启动一个容器。

kube minion(node)：

• kubelet
  负责node的管理，基本所有操作都靠它。

• kube-proxy
  每个node里的container都在一个私有网络中，kube-proxy的作用就是做一个反向代理，让访问者访问这个node的时候，可以转发到内部对应的container。

Flannel：

实现多台容器跨主机网络通信
（这里和minion放在一起）

etcd：

作为kubernetes的数据库，存储了k8s自身的信息、以及各种业务容器信息等。
存储flannel网络配置信息，供各节点协调。
（这里和master放在一起）

Tips: 以上组件介绍来自 方云麟的博客。

二、安装和配置

1. 配置 master kubernetes

• 安装

yum install -y docker kubernetes-master etcd 

• egrep -v ‘^#’ /etc/etcd/etcd.conf

ETCD_NAME=1.1.1.100
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="http://1.1.1.100:2380"
ETCD_LISTEN_CLIENT_URLS="http://1.1.1.100:2379","http://127.0.0.1:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://1.1.1.100:2380"
ETCD_INITIAL_CLUSTER="1.1.1.100=http://1.1.1.100:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://1.1.1.100:2379"

• cat /etc/kubernetes/config

# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=true"

# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"

# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow-privileged=false" 

# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=http://1.1.1.100:8080"

• cat /etc/kubernetes/apiserver

###
# kubernetes system config

# The following values are used to configure the kube-apiserver

# The address on the local server to listen to.
KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"
# The port on the local server to listen on.
# KUBE_API_PORT="--port=8080"
# Port minions listen on
# KUBELET_PORT="--kubelet-port=10250"
# Comma separated list of nodes in the etcd cluster
KUBE_ETCD_SERVERS="--etcd-servers=http://1.1.1.100:2379"
# Address range to use for services
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
# default admission control policies
KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota"
# Add your own!
KUBE_API_ARGS=""

• cat /etc/kubernetes/controller-manager

###
# The following values are used to configure the kubernetes controller-manager
# defaults from config and apiserver should be adequate
# Add your own!
KUBE_CONTROLLER_MANAGER_ARGS="--node-monitor-grace-period=10s --pod-eviction-timeout=10s"

• 启动 master,并查看状态

for service in etcd kube-apiserver kube-controller-manager kube-scheduler; do 
    systemctl enable $service
    systemctl restart $service
    systemctl status $service 
done

2. 配置 minion kubernetes

• 安装

yum install -y docker kubernetes-node flannel

• cat /etc/kubernetes/config

# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=true"

# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"

# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow-privileged=false" 

# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=http://1.1.1.100:8080"

• cat /etc/kubernetes/kubelet

# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
KUBELET_ADDRESS="--address=127.0.0.1"

# The port for the info server to serve on
# KUBELET_PORT="--port=10250"
# You may leave this blank to use the actual hostname，

#下面ip是个minion的ip
KUBELET_HOSTNAME="--hostname-override=1.1.1.101"  

# location of the api-server
KUBELET_API_SERVER="--api-servers=http://1.1.1.100:8080"

# pod infrastructure container
#KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"

# Add your own!
KUBELET_ARGS="--pod-infra-container-image=kubernetes/pause"

3. 配置flannel

• 【master】初始化flannel的etcd配置

etcdctl -C 1.1.1.100:2379 set /coreos.com/network/config '{ "Network": "10.1.0.0/16" }'

• 【minion】cat /etc/sysconfig/flanneld

# etcd url location.  Point this to the server where etcd runs
FLANNEL_ETCD="http://1.1.1.100:2379"

# etcd config key.  This is the configuration key that flannel queries

# For address range assignment
FLANNEL_ETCD_KEY="/coreos.com/network"

• 启动

for SERVICES in kube-proxy kubelet docker flanneld; do
    systemctl restart $SERVICES
    systemctl enable $SERVICES
    systemctl status $SERVICES 
done

• 查看状态
  

  minion

> ip a | grep flannel | grep inet
    inet 10.1.37.0/16 scope global flannel0

> ip a | grep flannel | grep inet
    inet 10.1.42.0/16 scope global flannel0

master

> etcdctl ls /coreos.com/network/subnets
/coreos.com/network/subnets/10.1.42.0-24
/coreos.com/network/subnets/10.1.37.0-24

三、验证

1. 在 master 上查看 node

kubectl get nodes
NAME        STATUS    AGE
1.1.1.101   Ready     4m
1.1.1.102   Ready     6m

**Tips**：

#删除node
kubectl delete node 2.2.2.101

2. 使用pods

• 创建 pods
  # 为了创建一个pod，我们需要在kubernetes master上面定义一个yaml 或者 json配置文件。然后使用kubectl命令创建pod

> mkdir -p /data/pods
> cd /data/pods/
> cat nginx.yaml

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  containers:
  - name: nginx
    # 这个地方我写的是我的私有仓库镜像，所以minion端的镜像配置也要改
    image: 1.1.1.100:500/nginx
    ports:
    - containerPort: 80

• 创建pod

kubectl create -f nginx.yaml

• 查看pod

> kubectl get pod nginx
NAME      READY     STATUS              RESTARTS   AGE
nginx     0/1       ContainerCreating   0          14s
> kubectl get pod -o wide
NAME      READY     STATUS              RESTARTS   AGE       NODE
nginx     0/1       ContainerCreating   0          9m        1.1.1.102

• 查看详情

> kubectl describe pod nginx
Name:       nginx
Namespace:  default
Node:       1.1.1.101/1.1.1.101
Start Time: Thu, 22 Sep 2016 17:46:18 +0800
Labels:     app=nginx
Status:     Pending
IP:     
Controllers:    <none>
Containers:
  nginx:
    Container ID:   
    Image:      1.1.1.100:500/nginx
    Image ID:       
    Port:       80/TCP
    QoS Tier:
      cpu:      BestEffort
      memory:       BestEffort
    State:      Waiting
      Reason:       ContainerCreating
    Ready:      False
    Restart Count:  0
    Environment Variables:
Conditions:
  Type      Status
  Ready     False 
No volumes.
Events:
  FirstSeen LastSeen    Count   From            SubobjectPath   Type        Reason      Message
  --------- --------    -----   ----            -------------   --------    ------      -------
  7s        7s      1   {default-scheduler }            Normal      Scheduled   Successfully assigned nginx to 1.1.1.102

# 查看详情可用于排错；

自此，kubernetes初步搭建完成，后续会介绍一些它的具体使用。。。

本文参考1：http://severalnines.com/blog/installing-kubernetes-cluster-minions-centos7-manage-pods-services
本文参考2：http://www.sunmite.com/linux/installing-kubernetes-cluster-on-centos7-to-manage-pods-and-services/#comment-564