CentOS7.2中使用Kubernetes（k8s)1.4.6源码搭建k8s容器集群环境

最新推荐文章于 2024-05-07 10:53:47 发布

Johnny_小鱼游水

最新推荐文章于 2024-05-07 10:53:47 发布

阅读量2.4k

点赞数 6

分类专栏： k8s Kubernetes container 容器 Docker 文章标签： centos 源码集群 kvm Kubernetes

本文链接：https://blog.csdn.net/yu_jinyu/article/details/53292399

版权

容器同时被 3 个专栏收录

2 篇文章 0 订阅

订阅专栏

Docker

2 篇文章 0 订阅

订阅专栏

k8s

1 篇文章 0 订阅

订阅专栏

一、相关准备工作

1.1、准备工作

准备至少两台已安装好CentOS7.2操作系统的物理机或者虚拟机（本文配置时使用的是三台KVM虚拟机）；
设置hostname命令：
hostnamectl set-hostname k8s-mst

角色	ip	hostname
Master	192.168.3.87	k8s-mst
Node	192.168.3.88	k8s-nod1
Node	192.168.3.89	k8s-nod2

为了避免和Docker的iptables产生冲突，需要关闭Node节点上的防火墙
systemctl stop firewalld
systemctl disable firewalld
为了让各个节点的时间保持一致，需要为所有节点安装NTP
yum -y install ntp
systemctl start ntpd
systemctl enable ntpd

1.2、编译Kubernetes源码(建议在Node节点上）

1.2.1、安装docker-engine

官方指导资料

使用以下方法可以安装较新版本

添加yum库

sudo tee /etc/yum.repos.d/docker.repo <<-'EOF'
[dockerrepo]
name=Docker Repository
baseurl=https://yum.dockerproject.org/repo/experimental/centos/7/
enabled=1
gpgcheck=1
gpgkey=https://yum.dockerproject.org/gpg
EOF

安装Docker-engine
sudo yum install -y docker-engine
运行Docker Daemon
sudo systemctl start docker
可以使用sudo systemctl status docker查看docker Daemon的运行状态

1.2.2、安装Golang

sudo yum install -y golang

1.2.3、下载Kubernetes

从Kubernetes的github下载源码
git clone https://github.com/kubernetes/kubernetes.git
或者直接下载相应版本的release包https://codeload.github.com/kubernetes/kubernetes/tar.gz/v1.4.6
如果使用git clone，下载完成后进入kubernetes文件夹，使用命令git checkout v1.4.6，下载release包则解压tar -xvf kubernetes-1.4.6.tar.gz进入kubernetes文件夹；

1.2.4、编译Kubernetes

修改hosts
由于在Kubernetes编译过程中需要pull谷歌容器库（gcr）中的相关镜像，故需要修改hosts进行翻墙，hosts文件参考：https://github.com/racaljk/hosts
修改运行平台配置参数
根据自己的运行平台（linux/amd64)修改hack/lib/golang.sh，把KUBE_SERVER_PLATFORMS，KUBE_CLIENT_PLATFORMS和KUBE_TEST_PLATFORMS中除linux/amd64以外的其他平台注释掉，以此来减少编译所用时间
编译源码
在Kubernetes根目录下运行命令make release-skip-tests，编译耗时相对较长
编译成功之后，可执行文件在文件夹“_output”中

二、Master配置工作

2.1、安装ectd并修改配置文件

2.1.1、安装必要软件etcd

yum -y install etcd

2.1.2、修改etcd的配置文件/etc/etcd/etcd.conf

ETCD_NAME=default
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"

2.1.3、运行etcd

sudo systemctl start etcd

2.1.4、配置etcd中的网络

etcdctl mk /k8s/network/config '{"Network":"172.17.0.0/16"}'

2.2、kubernetes环境配置

2.2.1、复制命令（可执行文件）

将位于_output/release-stage/server/linux-amd64/kubernetes/server/bin/目录下的kube-apiserver、kube-controller-manager、kube-scheduler、kubectl复制到Master节点的/usr/bin/目录下

2.2.2、创建相应的service文件以及配置文件（shell脚本）

根据自己的的配置修改MASTER_ADDRESS和ETCD_SERVERS

#!/bin/bash
# Copyright 2016 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

MASTER_ADDRESS=${1:-"192.168.3.87"}
ETCD_SERVERS=${2:-"http://192.168.3.87:2379"}
SERVICE_CLUSTER_IP_RANGE=${3:-"10.254.0.0/16"}
ADMISSION_CONTROL=${4:-"NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"}

cat <<EOF >/etc/kubernetes/config
# --logtostderr=true: log to standard error instead of files
KUBE_LOGTOSTDERR="--logtostderr=true"

# --v=0: log level for V logs
KUBE_LOG_LEVEL="--v=0"

# --allow-privileged=false: If true, allow privileged containers.
KUBE_ALLOW_PRIV="--allow-privileged=false"

# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=${MASTER_ADDRESS}:8080"
EOF

cat <<EOF >/etc/kubernetes/apiserver
# --insecure-bind-address=127.0.0.1: The IP address on which to serve the --insecure-port.
KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"

# --insecure-port=8080: The port on which to serve unsecured, unauthenticated access.
KUBE_API_PORT="--insecure-port=8080"

# --kubelet-port=10250: Kubelet port
NODE_PORT="--kubelet-port=10250"

# --etcd-servers=[]: List of etcd servers to watch (http://ip:port), 
# comma separated. Mutually exclusive with -etcd-config
KUBE_ETCD_SERVERS="--etcd-servers=${ETCD_SERVERS}"

# --advertise-address=<nil>: The IP address on which to advertise 
# the apiserver to members of the cluster.
KUBE_ADVERTISE_ADDR="--advertise-address=${MASTER_ADDRESS}"

# --service-cluster-ip-range=<nil>: A CIDR notation IP range from which to assign service cluster IPs. 
# This must not overlap with any IP ranges assigned to nodes for pods.
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=${SERVICE_CLUSTER_IP_RANGE}"

# --admission-control="AlwaysAdmit": Ordered list of plug-ins 
# to do admission control of resources into cluster. 
# Comma-delimited list of: 
#   LimitRanger, AlwaysDeny, SecurityContextDeny, NamespaceExists, 
#   NamespaceLifecycle, NamespaceAutoProvision,
#   AlwaysAdmit, ServiceAccount, ResourceQuota, DefaultStorageClass
KUBE_ADMISSION_CONTROL="--admission-control=${ADMISSION_CONTROL}"

# Add your own!
KUBE_API_ARGS=""
EOF

KUBE_APISERVER_OPTS="   \${KUBE_LOGTOSTDERR}         \\
                        \${KUBE_LOG_LEVEL}           \\
                        \${KUBE_ETCD_SERVERS}        \\
                        \${KUBE_API_ADDRESS}         \\
                        \${KUBE_API_PORT}            \\
                        \${NODE_PORT}                \\
                        \${KUBE_ADVERTISE_ADDR}      \\
                        \${KUBE_ALLOW_PRIV}          \\
                        \${KUBE_SERVICE_ADDRESSES}   \\
                        \${KUBE_ADMISSION_CONTROL}   \\
                        \${KUBE_API_ARGS}"


cat <<EOF >/usr/lib/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes
After=network.target
After=etcd.service

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/apiserver
ExecStart=/usr/bin/kube-apiserver ${KUBE_APISERVER_OPTS}
Restart=on-failure
Type=notify
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

cat <<EOF >/etc/kubernetes/controller-manager
###
# The following values are used to configure the kubernetes controller-manager

# defaults from config and apiserver should be adequate

# Add your own!
KUBE_CONTROLLER_MANAGER_ARGS=""
EOF

KUBE_CONTROLLER_MANAGER_OPTS="  \${KUBE_LOGTOSTDERR} \\
                                \${KUBE_LOG_LEVEL}   \\
                                \${KUBE_MASTER}      \\
                                \${KUBE_CONTROLLER_MANAGER_ARGS}"

cat <<EOF >/usr/lib/systemd/system/kube-controller-manager.service
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/kubernetes/kubernetes

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/controller-manager
ExecStart=/usr/bin/kube-controller-manager ${KUBE_CONTROLLER_MANAGER_OPTS}
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

cat <<EOF >/etc/kubernetes/scheduler
###
# kubernetes scheduler config

# Add your own!
KUBE_SCHEDULER_ARGS=""
EOF

KUBE_SCHEDULER_OPTS="   \${KUBE_LOGTOSTDERR}     \\
                        \${KUBE_LOG_LEVEL}       \\
                        \${KUBE_MASTER}          \\
                        \${KUBE_SCHEDULER_ARGS}"

cat <<EOF >/usr/lib/systemd/system/kube-scheduler.service
[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/kubernetes/kubernetes

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/scheduler
ExecStart=/usr/bin/kube-scheduler ${KUBE_SCHEDULER_OPTS}
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload

2.2.3、运行相应的Kubernetes命令（shell 脚本）

for svc in etcd kube-apiserver kube-controller-manager kube-scheduler; do 
    systemctl restart $svc
    systemctl enable $svc
    systemctl status $svc 
done

三、Node配置工作

3.1、安装flannel并修改配置文件

3.1.1、安装必要软件flannel

yum -y install flannel

3.1.2、修改flannel的配置文件/etc/sysconfig/flanneld

FLANNEL_ETCD="http://192.168.3.87:2379"
FLANNEL_ETCD_KEY="/k8s/network"

3.1.3、运行flannel

systemctl restart flanneld
systemctl enable flanneld
systemctl status flanneld

3.1.4、上传网络配置

创建一个config.json文件，内容如下：

{
"Network": "172.17.0.0/16",
"SubnetLen": 24,
"Backend": {
     "Type": "vxlan",
     "VNI": 7890
     }
 }

然后将配置上传到etcd服务器上：
curl -L http://192.168.3.87:2379/v2/keys/k8s/network/config -XPUT --data-urlencode value@config.json

3.2、kubernetes环境配置

3.2.1、复制命令（可执行文件）

将位于_output/release-stage/server/linux-amd64/kubernetes/server/bin/目录下的kube-proxy、kubelet 复制到Node节点的/usr/bin/目录下

3.2.2、创建相应的service文件以及配置文件（shell脚本）

根据自己的的配置修改MASTER_ADDRESS和NODE_HOSTNAME

#!/bin/bash
# Copyright 2016 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

MASTER_ADDRESS=${1:-"192.168.3.87"}
NODE_HOSTNAME=${2:-"k8s-nod"}

cat <<EOF >/etc/kubernetes/config
# --logtostderr=true: log to standard error instead of files
KUBE_LOGTOSTDERR="--logtostderr=true"

# --v=0: log level for V logs
KUBE_LOG_LEVEL="--v=0"

# --allow-privileged=false: If true, allow privileged containers.
KUBE_ALLOW_PRIV="--allow-privileged=false"

# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=${MASTER_ADDRESS}:8080"
EOF

cat <<EOF >/etc/kubernetes/proxy
###
# kubernetes proxy config

# default config should be adequate

# Add your own!
KUBE_PROXY_ARGS=""
EOF

KUBE_PROXY_OPTS="   \${KUBE_LOGTOSTDERR} \\
                    \${KUBE_LOG_LEVEL}   \\
                    \${KUBE_MASTER}    \\
                    \${KUBE_PROXY_ARGS}"

cat <<EOF >/usr/lib/systemd/system/kube-proxy.service
[Unit]
Description=Kubernetes Proxy
After=network.target

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/kube-proxy
ExecStart=/usr/bin/kube-proxy ${KUBE_PROXY_OPTS}
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

cat <<EOF >/etc/kubernetes/kubelet
# --address=0.0.0.0: The IP address for the Kubelet to serve on (set to 0.0.0.0 for all interfaces)
KUBELET__ADDRESS="--address=0.0.0.0"

# --port=10250: The port for the Kubelet to serve on. Note that "kubectl logs" will not work if you set this flag.
KUBELET_PORT="--port=10250"

# --hostname-override="": If non-empty, will use this string as identification instead of the actual hostname.
KUBELET_HOSTNAME="--hostname-override=${NODE_HOSTNAME}"

# --api-servers=[]: List of Kubernetes API servers for publishing events, 
# and reading pods and services. (ip:port), comma separated.
KUBELET_API_SERVER="--api-servers=${MASTER_ADDRESS}:8080"

# pod infrastructure container
KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"

# Add your own!
KUBELET_ARGS=""
EOF

KUBE_PROXY_OPTS="   \${KUBE_LOGTOSTDERR}     \\
                    \${KUBE_LOG_LEVEL}       \\
                    \${KUBELET__ADDRESS}         \\
                    \${KUBELET_PORT}            \\
                    \${KUBELET_HOSTNAME}        \\
                    \${KUBELET_API_SERVER}   \\
                    \${KUBE_ALLOW_PRIV}      \\
                    \${KUBELET_POD_INFRA_CONTAINER}\\
                    \${KUBELET_ARGS}"

cat <<EOF >/usr/lib/systemd/system/kubelet.service
[Unit]
Description=Kubernetes Kubelet
After=docker.service
Requires=docker.service

[Service]
WorkingDirectory=/var/lib/kubelet
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/kubelet
ExecStart=/usr/bin/kubelet ${KUBE_PROXY_OPTS}
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload

3.2.3、运行相应的Kubernetes命令（shell脚本）

for svc in kube-proxy kubelet docker; do 
    systemctl restart $svc
    systemctl enable $svc
    systemctl status $svc
done

四、验证配置以及创建dashboard

4.1、验证环境配置

在Master节点运行命令kubectl get nodes，输出信息如下：

[root@k8s-mst ~]# kubectl get nodes 
 NAME      STATUS    AGE
 nod1       Ready     3h 
 nod2       Ready     3h

4.2、搭建dashboard

4.2.1、创建命名空间（namespace）kube-system

创建文件kubernetes-namespace.jason，内容如下：

{
  "kind": "Namespace",
  "apiVersion": "v1",
  "metadata": {
    "name": "kube-system"
  }
}

使用命令kubectl create -f kubernetes-namespace.jason创建kube-system命名空间。

4.2.2、创建dashboard

创建kubernetes-dashboard.yaml文件

# Configuration to deploy release version of the Dashboard UI.
# Example usage: kubectl create -f <this_file>

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  labels:
    app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kubernetes-dashboard
  template:
    metadata:
      labels:
        app: kubernetes-dashboard
      # Comment the following annotaion if Dashboard must not be deployed on master
      annotations:
        scheduler.alpha.kubernetes.io/tolerations: |
          [
            {
              "key": "dedicated",
              "operator": "Equal",
              "value": "master",
              "effect": "NoSchedule"
            }
          ]
    spec:
      containers:
      - name: kubernetes-dashboard
        image: gcr.io/google_containers/kubernetes-dashboard-amd64:v1.4.2
        imagePullPolicy: Always
        ports:
        - containerPort: 9090
          protocol: TCP
        args:
          # Uncomment the following line to manually specify Kubernetes API server Host
          # If not specified, Dashboard will attempt to auto discover the API server and connect
          # to it. Uncomment only if the default does not work.
          - --apiserver-host=http://192.168.3.87:8080
        livenessProbe:
          httpGet:
            path: /
            port: 9090
          initialDelaySeconds: 30
          timeoutSeconds: 30
---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kube-system
spec:
  type: NodePort
  ports:
  - port: 80
    targetPort: 9090
  selector:
    app: kubernetes-dashboard

注意：

修改 –apiserver-host=http://192.168.3.87:8080；
其中 “ image: gcr.io/google_containers/kubernetes-dashboard-amd64:v1.4.2 ”，使用的是谷歌镜像库，需要Node节点翻墙才会可能正常创建。

使用命令kubectl create -f kubernetes-dashboard.yaml创建kubernetes-dashboard Deployment和Service。
如无法获取相应的镜像创建之后，使用命令kubectl get pod --namespace="kube-system"查看会显示如下结果：

[root@mst ~]# kubectl get pod --namespace="kube-system"
NAME                                  READY     STATUS         RESTARTS   AGE
kubernetes-dashboard-47291540-lcuox   0/1       ErrImagePull   0          1m

搭建自己的镜像库，可参考【Docker实战】Registry & Portus搭建详解 - 龍隐 - 博客园 http://www.cnblogs.com/xcloudbiz/articles/5497037.html

4.2.3、使用dashboard

成功运行之后：

[root@k8s-mst ~]# kubectl get pod --namespace="kube-system"
NAME                                    READY     STATUS    RESTARTS   AGE
kubernetes-dashboard-3856900779-226mr   1/1       Running   0          2m

[root@k8s-mst ~]# kubectl describe  pod kubernetes-dashboard-3856900779-226mr --namespace="kube-system"
Name:       kubernetes-dashboard-3856900779-226mr
Namespace:  kube-system
Node:       nod1/192.168.3.91
Start Time: Tue, 22 Nov 2016 20:33:04 +0800
Labels:     app=kubernetes-dashboard
        pod-template-hash=3856900779
Status:     Running
IP:     172.18.0.2
Controllers:    ReplicaSet/kubernetes-dashboard-3856900779
Containers:
  kubernetes-dashboard:
    Container ID:   docker://d36cff522129c73f0de370857124697659662c99d370af548a1367604bac7014
    Image:      gcr.io/google_containers/kubernetes-dashboard-amd64:v1.4.2
    Image ID:       docker://sha256:c0e4ba8968ee756368cbe5f64f39b0ef8e128de90d0bdfe1d040f0773055e68a
    Port:       9090/TCP
    Args:
      --apiserver-host=http://192.168.3.87:8080
    State:          Running
      Started:          Tue, 22 Nov 2016 20:35:00 +0800
    Ready:          True
    Restart Count:      0
    Liveness:           http-get http://:9090/ delay=30s timeout=30s period=10s #success=1 #failure=3
    Volume Mounts:      <none>
    Environment Variables:  <none>
Conditions:
  Type      Status
  Initialized   True 
  Ready     True 
  PodScheduled  True 
No volumes.
QoS Class:  BestEffort
Tolerations:    dedicated=master:Equal:NoSchedule
Events:
  FirstSeen LastSeen    Count   From            SubobjectPath               Type        Reason          Message
  --------- --------    -----   ----            -------------               --------    ------          -------
  4m        4m      1   {default-scheduler }                        Normal      Scheduled       Successfully assigned kubernetes-dashboard-3856900779-226mr to nod1
  3m        3m      1   {kubelet nod1}      spec.containers{kubernetes-dashboard}   Normal      Pulling         pulling image "gcr.io/google_containers/kubernetes-dashboard-amd64:v1.4.2"
  3m        2m      2   {kubelet nod1}                          Warning     MissingClusterDNS   kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy.
  2m        2m      1   {kubelet nod1}      spec.containers{kubernetes-dashboard}   Normal      Pulled          Successfully pulled image "gcr.io/google_containers/kubernetes-dashboard-amd64:v1.4.2"
  2m        2m      1   {kubelet nod1}      spec.containers{kubernetes-dashboard}   Normal      Created         Created container with docker id d36cff522129; Security:[seccomp=unconfined]
  2m        2m      1   {kubelet nod1}      spec.containers{kubernetes-dashboard}   Normal      Started         Started container with docker id d36cff522129

[root@mst ~]# kubectl describe service  kubernetes-dashboard  --namespace="kube-system"
Name:           kubernetes-dashboard
Namespace:      kube-system
Labels:         app=kubernetes-dashboard
Selector:       app=kubernetes-dashboard
Type:           NodePort
IP:         10.254.196.154
External IPs:       192.168.3.87
Port:           <unset> 80/TCP
NodePort:       <unset> 31437/TCP
Endpoints:      172.18.0.2:9090
Session Affinity:   None

运行成功之后，就可以使用浏览器访问192.168.3.89:31437使用dashboard。

Johnny_小鱼游水

关注

6
点赞
踩
1

收藏

觉得还不错? 一键收藏
0
评论
CentOS7.2中使用Kubernetes（k8s)1.4.6源码搭建k8s容器集群环境

一、相关准备工作1、准备工作准备至少两台已安装好CentOS7.2操作系统的物理机或者虚拟机（本文配置时使用的是三台KVM虚拟机）；设置hostname命令： hostnamectl set-hostname k8s-mst 角色 ip hostname Master 192.168.3.87 k8s-mst Node 192.168.3.88 k8s-nod
复制链接

扫一扫