Kubernetes1.3 - K8S安装
1 Kubernetes规划
1.1 规划
Master 192.168.1.11 apiserver、controller-manager、scheduler
etcd 192.168.1.11 etcd
NODE 192.168.1.12 Kubernetes+docker
NODE 192.168.1.13 Kubernetes+docker
版本:Kubernetes1.3
1.2 前期准备
(以下步骤在所有服务器上执行)
1.2.1 停止防火墙
systemctl stop firewalld.service
systemctl disable firewalld.service
1.2.2 配置yum源
挂载系统的安装ISO文件
(rhel6.5.iso)
mount /dev/sr0 /mnt
cat >> /etc/yum.repos.d/rhel-source.repo<< EOF
[rhel-source]
name=Red Hat Enterprise Linux $releasever - $basearch - Source
baseurl=file:///mnt
enabled=1
gpgcheck=1
gpgkey=file:///mnt/RPM-GPG-KEY-redhat-release
EOF
1.2.3 安装IPTABLES
yum install -y iptables-services
systemctl start iptables.service
systemctl enable iptables.service
2 安装Docker
(在Minion节点安装docker)
Docker文档地址:
https://docs.docker.com/engine/installation/linux/rhel/
2.1 配置安装源
配置安装源(方法一)
cat >> /etc/yum.repos.d/docker.repo <<-EOF
[dockerrepo]
name=Docker Repository
baseurl=https://yum.dockerproject.org/repo/main/centos/7
enabled=1
gpgcheck=1
gpgkey=https://yum.dockerproject.org/gpg
EOF
配置安装源(方法二)
cat >> /etc/yum.repos.d/CentOS7-Base-163.repo<< EOF
[base]
name=CentOS-$releasever - Base - 163.com
baseurl=http://mirrors.163.com/centos/7/os/$basearch/
gpgcheck=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-7
#released updates
[updates]
name=CentOS-$releasever - Updates - 163.com
baseurl=http://mirrors.163.com/centos/7/updates/$basearch/
gpgcheck=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-7
#additional packages that may be useful
[extras]
name=CentOS-$releasever - Extras - 163.com
baseurl=http://mirrors.163.com/centos/7/extras/$basearch/
gpgcheck=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-7
EOF
2.2 安装
yum install docker
启动docker服务。
service docker start
#关闭防火墙
service firewalld stop
确认docker是否安装成功。
docker run hello-world
3 安装KUBERNETES
3.1 安装配置ETCD
3.1.1 下载安装ETCD
ETCD官方文档
https://github.com/coreos/etcd/releases/
etcd无需安装,下载即可使用
curl -L https://github.com/coreos/etcd/releases/download/v3.0.2/etcd-v3.0.2-linux-amd64.tar.gz -o etcd-v3.0.2-linux-amd64.tar.gz
tar xzvf etcd-v3.0.2-linux-amd64.tar.gz
cd etcd-v3.0.2-linux-amd64
./etcd --version
mkdir -p /opt/etcd/bin /opt/etcd/data
./etcd
mkdir /opt/etcd/data
3.1.2 配置ETCD
可参考解压包里的官方文档安装配置ETCD,路径如下:
more /root/etcd-v3.0.2-linux-amd64/Documentation/demo.md
cat >>/opt/etcd/ku_etcd.sh <<eof </eof<>
#!/bin/bash
etcd_data_dir=/var/lib/etcd/
mkdir -p ${etcd_data_dir}
cat </opt/kubernetes/cfg/etcd.conf
# [member]
ETCD_NAME=default
ETCD_DATA_DIR="${etcd_data_dir}/default.etcd"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:4001"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.1.11:2379,http://192.168.1.11:4001"
EOF
3.1.3 开机启动ETCD
cat </usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
[Service]
ExecStart=/opt/etcd/bin/etcd \
--name etcd-1 \
--initial-advertise-peer-urls http://192.168.1.11:2380 --listen-peer-urls http://192.168.1.11:2380 \
--advertise-client-urls http://192.168.1.11:2379 --listen-client-urls http://192.168.1.11:2379 \
--initial-cluster etcd-1=http://192.168.1.11:2380 \
--initial-cluster-state new --initial-cluster-token token-01
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable etcd
systemctl start etcd
3.1.4 检查ETCD
etcdctl --endpoints=192.168.1.11:2379 member list
etcdctl member list
etcdctl mk key value
etcdctl get key
在其他主机上,验证远程连接的正确性,是否可以正确设置和获取
etcdctl -endpoints http://192.168.1.11:4001 get key
etcdctl -endpoints http://192.168.1.11:4001 mk key2 value2
etcdctl -endpoints http://192.168.1.11:4001 get key2
################# kubernetes ##############################################
wget https://github.com/kubernetes/kubernetes/releases/download/v1.3.0/kubernetes.tar.gz
mkdir -p /opt/kubernetes/bin
tar xf kubernetes.tar.gz
cd /root/kubernetes/server/
tar xf kubernetes-server-linux-amd64.tar.gz
Master:
cp kubernetes/server/bin/kube* /opt/kubernetes/bin/
minion:
而kubelet,kube-proxy则复制到minion1和minion2
cd /root/kubernetes/server/
scp kubelet kube-proxy* 192.168.1.12:/opt/kubernetes/bin/
scp kubelet kube-proxy* 192.168.1.13:/opt/kubernetes/bin/
3.1.5 配置环境变量
Master配置环境变量:
PATH=$PATH:$HOME/bin:/opt/etcd/bin/:/opt/kubernetes/bin
minion配置环境变量:
PATH=$PATH:$HOME/bin:/opt/kubernetes/bin
验证版本信息:
/opt/kubernetes/bin/kubectl version
4 下载KUBERNETES
4.1 下载并解压
wget https://github.com/kubernetes/kubernetes/releases/download/v1.3.0/kubernetes.tar.gz
mkdir -p /opt/kubernetes/bin
tar xf kubernetes.tar.gz
cd /root/kubernetes/server/
tar xf kubernetes-server-linux-amd64.tar.gz
4.2 Master节点
复制相关程序到Master节点
cp kubernetes/server/bin/kube* /opt/kubernetes/bin/
4.3 NODE节点
而kubelet,kube-proxy则复制到minion1和minion2
cd /root/kubernetes/server/
scp kubelet kube-proxy* 192.168.1.12:/opt/kubernetes/bin/
scp kubelet kube-proxy* 192.168.1.13:/opt/kubernetes/bin/
5 安装KUBERNETES的MASTER节点
5.1 安装APISERVER
解压目录有官方参考安装脚本:
/root/kubernetes/cluster/centos/master/scripts/apiserver.sh
执行apiserver.sh脚本
cd /root/kubernetes/cluster/centos/master/scripts
sh apiserver.sh “192.168.1.11” "http://8.8.8.18:4001" "10.10.10.0/24"
5.2 安装controller-manager
解压目录有官方参考安装脚本:
/root/kubernetes/cluster/centos/master/scripts/controller-manager.sh
执行controller-manager.sh脚步
cd /root/kubernetes/cluster/centos/master/scripts/
sh controller-manager.sh 192.168.8.11
5.3 安装scheduler
解压目录有官方参考安装脚本:
/root/kubernetes/cluster/centos/master/scripts/ scheduler.sh
cd /root/kubernetes/cluster/centos/master/scripts/
sh scheduler.sh 192.168.1.11
6 安装配置KUBERNETES的NODE节点
6.1 FLANNEL安装与配置
6.1.1 FLANNEL安装
wget -c https://github.com/coreos/flannel/releases/download/v0.5.5/flannel-0.5.5-linux-amd64.tar.gz
tar zxvf flannel-0.5.5-linux-amd64.tar.gz
cd flannel-0.5.5-linux-amd64
cp flannel /opt/kubernetes/bin/
6.1.2 FLANNEL配置
解压目录有官方参考安装脚本:
/root/kubernetes/cluster/centos/node/scripts/flannel.sh
Flannel.sh使用到了mk-docker-opts.sh和remove-docker0.sh这两个文件在
/root/kubernetes/cluster/centos/node/bin/mk-docker-opts.sh
/root/kubernetes/cluster/centos/node/bin/remove-docker0.sh
复制到/opt/kubernetes/bin/目录
cp /root/kubernetes/cluster/centos/node/bin/* /opt/kubernetes/bin/
快速配置flannel
sh flannel.sh "http://192.168.1.11:4001" “172.16.18.0/16”
6.2 配置kubelet
解压目录有官方参考安装脚本:
/root/kubernetes/cluster/centos/node/scripts/kubelet.sh
cd /root/kubernetes/cluster/centos/node/scripts
NODE1:
sh kubelet.sh “192.168.1.11” “192.168.1.12”
NODE2:
sh kubelet.sh “192.168.1.11” “192.168.1.13”
说明:192.168.1.11是APISERVER的地址,
192.168.1.12是NODE的IP地址
6.3 配置proxy
解压目录有官方参考安装脚本:
/root/kubernetes/cluster/centos/node/scripts/proxy.sh
cd /root/kubernetes/cluster/centos/node/scripts
NODE1:
sh proxy.sh “192.168.1.11” “192.168.1.12”
NODE2:
sh proxy.sh “192.168.1.11” “192.168.1.13”
说明:192.168.1.11是APISERVER的地址,
192.168.1.12是NODE的IP地址
6.4 配置DOCKER
解压目录有官方参考安装脚本:
/root/kubernetes/cluster/centos/node/scripts/docker.sh
cd /root/kubernetes/cluster/centos/node/scripts
NODE节点治:
sh docker.sh
至此,kubernetes已安装完毕
7 使用KUBERNETES
kubernetes
Master:
/opt/kubernetes/bin/kubectl -s 192.168.1.11:8080 create -f tomcat.yaml
查看:
查看所有NODE是否正常
kubectl -s 192.168.1.11:8080 get no
kubectl -s 192.168.1.11:8080 describe po tomcat
查看pod、replication controller、service和endpoint
kubectl -s 192.168.1.11:8080 get po
kubectl -s 192.168.1.11:8080 get rc
kubectl -s 192.168.1.11:8080 get svc
kubectl -s 192.168.1.11:8080 get ep
kubectl get rc,pods
kubectl describe pod tomcat111
kubectl logs tomcat111-3506222467-pnx1l
删除:
kubectl -s 192.168.1.11:8080 delete -f tomcat.yaml
kubectl get po # 查看目前所有的pod
kubectl get rs # 查看目前所有的replica set
kubectl get deployment # 查看目前所有的deployment
kubectl describe po my-nginx # 查看my-nginx pod的详细状态
kubectl describe rs my-nginx # 查看my-nginx replica set的详细状态
kubectl describe deployment my-nginx # 查看my-nginx deployment的详细状态
kubectl run foo2 --image=hello-world
kubectl delete deployment foo2
kubernetes的监控采用了cAdvisor组件。因为kubernetes中记录了容器的信息(但是没有记录容器-网卡的映射关系),所以运行在节点上的cAdvisor不需要通过docker stats去获取容器的cpu和内存使用数据。
而网络IO数据呢?
我们知道k8s部署运行一个容器是会先生成一个pause容器。
是的,网络IO都记录在pause容器中。这里大家可以在自己的k8s环境中验证。
所以只要我们获取某容器对应的pause容器的containerID,我们就可以用如上的方式去抓到网络IO。
因为cAdvisor并不是为k8s专门设计的,不会特地在获取网络IO时去遍历查找容器对应的pause容器。所以当前的cAdvisor没有办法获取容器的网络IO。
所以如果在使用k8s集群,想要加入网络IO监控功能的时候,可以从k8s自带的cAdvisor入手。
问题(1) kubectl get rc,pods 一直处于“ ContainerCreating ”状态
kubectl get rc,pods
原因:k8s默认从gcr.io/google_containers拉去镜像,国内网络无法访问。
给每个minion下载Google pause镜像。之后的工作会用到gcr.io/google_containers/pause镜像,
国内用户由于GFW问题总是会出错,所以我们先从docker.io下载pause镜像,并设置tag。
minion节点:
docker pull kubernetes/pause
docker tag kubernetes/pause gcr.io/google_containers/pause-amd64:3.0
8 使用kubernetes pod复制器
########################## nginx pod复制器 ######################
cat > nginx-rc.yaml <<eof </eof<>
apiVersion: v1
kind: ReplicationController
metadata:
name: nginx-controller
spec:
replicas: 2
selector:
name: nginx
template:
metadata:
labels:
name: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
EOF
我们定义了一个nginx pod复制器,复制份数为2,我们使用nginx docker镜像。
执行下面的操作创建nginx pod复制器:
kubectl create -f nginx-rc.yaml
kubectl get service
########################## Service ##########################
Service的type有ClusterIP和NodePort之分,缺省是ClusterIP,这种类型的Service只能在集群内部访问。下表是本文用的配置文件:
cat >nginx-service-clusterip.yaml <<eof </eof<>
apiVersion: v1
kind: Service
metadata:
name: nginx-service-clusterip
spec:
ports:
- port: 8001
targetPort: 80
protocol: TCP
selector:
name: nginx
EOF
kubectl create -f ./nginx-service-clusterip.yaml services/nginx-service
kubectl get service
kubectl describe service nginx-service-clusterip
############################### #######################
部署外部可访问的nginx service
下面我们创建NodePort类型的Service,这种类型的Service在集群外部是可以访问。下表是本文用的配置文件:
cat >nginx-service-nodeport.yaml<<eof </eof<>
apiVersion: v1
kind: Service
metadata:
name: nginx-service-nodeport
spec:
ports:
- port: 8000
targetPort: 80
protocol: TCP
type: NodePort
selector:
name: nginx
EOF
kubectl create -f ./nginx-service-nodeport.yaml
kubectl get service
kubectl describe service nginx-service-nodeport
9 SKYDNS
9.1 SKYDNS简介
SKYDNS 已经被淘汰,建议用KUBE-DNS,请参考下一章节
DNS Server包含3部分:
- skyDNS: 提供DNS解析服务
- etcd:用于skyDNS的存储
- kube2sky:连接Kubernetes和skyDNS
注意:skydns服务使用的clusterIP需要我们指定一个固定的IP地址,每个Node的Kubelet进程都将使用这个IP地址,不能通过Kubernetes自动分配。
另外,这个IP地址需要在kube-apiserver启动参数–service-cluster-ip-range指定的IP地址范围内。
9.2 SKYDNS安装
(本章节的操作在APISERVER节点下)
可以参考这篇文章配置SKYDNS,也可以使用DOCKER容器的方式部署SKYDNS。
http://www.tuicool.com/articles/yeIJNjJ
https://github.com/GoogleCloudPlatform/kubernetes/cluster/addons/dns/kube2sky
(kube2sky的地址已经不存在,我找了很久也没找到)
https://github.com/gravitational/kube2sky
本文使用DOCKER容器的方式部署SKYDNS。
Kubernetes.tar.gz里有创建SKYDNS的YAML文件,路径如下:
cp /root/kubernetes/docs/getting-started-guides/coreos/azure/addons/skydns-rc.yaml /root/script/
cp /root/kubernetes/docs/getting-started-guides/coreos/azure/addons/skydns-svc.yaml /root/script/
cd /root/script/
修改skydns-rc.yaml
kube_master_url=http://192.168.1.11:8080
domain=kube.local
kube_master_url为apiserver的IP
domain为DNS的域名
为DNS服务中设置的域名。
修改skydns-rc.yaml
clusterIP: 10.10.10.100
(DNS的IP)
9.3 NODE节点下载相关镜像
在所有的NODE节点下载相关镜像(需翻墙下载)
docker pull gcr.io/google_containers/etcd:2.0.9
docker pull gcr.io/google_containers/kube2sky:1.11
docker pull gcr.io/google_containers/skydns:2015-03-11-001
docker pull gcr.io/google_containers/exechealthz:1.0
docker pull gcr.io/google_containers/pause-amd64
备注:gcr.io/google_containers/pause-amd64是kubernetes的基础镜像,必须要下载
9.4 创建DNS的POD和SERVICE
(在APISERVER服务器上执行)
创建DNS
kubectl create –f skydns-rc.yaml
kubectl create –f kydns-svc.yaml
查看状态
kubectl get rc --namespace=kube-system
kubectl get pods --namespace=kube-system
kubectl get services --namespace=kube-system
9.5 NODE节点修改Kubelet服务的启动参数
修改每台Node上修改Kubelet服务的启动参数:
--cluster_dns=10.10.10.100,为DNS服务的ClusterIP地址;
--cluster_domain=cluster.local,为DNS服务中设置的域名。
然后重启Kubelet服务。
systemctl daemon-reload
systemctl restart kubelet.service
下载测试DNS的busybox镜像
docker pull busybox
测试
kubectl exec busybox -- nslookup kube-dns.kube-system
如果解析不成功,可以看看etcd是否有这个DNS的KEY
kubectl exec busybox --namespace=default -- nslookup kube-local.kube-system
docker exec 725470b1d3f5 etcdctl ls /
10 KUBE-DNS
Kubernetes V1.3.x新增了kube-dns来替换skydns,部署相当简单。
在master主机部署kube-dns步骤:
10.1 新建kube-dns配置文件
# vi /etc/kubernetes/cfg/kube-dns
# kubernetes kube-dns config
KUBE_DNS_PORT="--dns-port=53"
KUBE_DNS_DOMAIN="--domain=bstk8s.com"
KUBE_DNS_MASTER=--kube-master-url="http://192.168.1.11:8080"
KUBE_DNS_ARGS=""
10.2 新建kube-dns.service配置文件
# vi /usr/lib/systemd/system/kube-dns.service
[Unit]
Description=Kubernetes Kube-dns Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=kube-apiserver.service
Requires=kube-apiserver.service
[Service]
WorkingDirectory=/var/lib/kube-dns
EnvironmentFile=-/etc/kubernetes/cfg/kube-dns
ExecStart=/opt/kubernetes/bin/kube-dns \
$KUBE_DNS_PORT \
$KUBE_DNS_DOMAIN \
$KUBE_DNS_MASTER \
$KUBE_DNS_ARGS
Restart=on-failure
[Install]
WantedBy=multi-user.target
创建目录
mkdir -p /var/lib/kube-dns
配置service自动启动
systemctl enable kube-dns
systemctl restart kube-dns
systemctl status kube-dns
10.3 所有Node节点主机修改kubelet文件
# vi /opt/kubernetes/cfg/kubelet
....
# Add your own!
KUBELET_ARGS="--cluster-dns=192.168.1.11 \
--cluster-domain=bstk8s.com"
10.4 修改resolv.conf,添加域名解析服务器地址
[root@k8s-master ~]# vi /etc/resolv.conf
# Generated by NetworkManager
search default.svc.bstk8s.com svc.bstk8s.com bstk8s.com
nameserver 192.168.1.11
nameserver 8.8.8.8
nameserver 114.114.114.114
10.5 DNS测试
查看所有service
kubectl get svc --all-namespaces
kubectl exec busybox -- nslookup kube-system
如果解析不成功,可以看看etcd是否有这个DNS的KEY
在ETCD所在主机上:
etcdctl ls /
11 cAdvisor
Google的 cAdvisor 是另一个知名的开源容器监控工具。
只需在宿主机上部署cAdvisor容器,用户就可通过Web界面或REST服务访问当前节点和容器的性能数据(CPU、内存、网络、磁盘、文件系统等等),非常详细。
默认cAdvisor是将数据缓存在内存中,数据展示能力有限;它也提供不同的持久化存储后端支持,可以将监控数据保存、汇总到Google BigQuery、InfluxDB或者Redis之上。
新的Kubernetes版本里,cadvior功能已经被集成到了kubelet组件中
需要注意的是,cadvisor的web界面,只能看到单前物理机上容器的信息,其他机器是需要访问对应ip的url,数量少时,很有效果,当数量多时,比较麻烦,所以需要把cadvisor的数据进行汇总、展示,就看“cadvisor+influxdb+grafana”
Cadvisor,只需要 在kubelet命令中,启用Cadvisor,和配置相关信息 ,即可
(1)在/opt/kubernetes/cfg/kubelet 加入如下 参数:
CADVISOR=' --cadvisor-port=4194 --storage-driver-db="cadvisor" --storage-driver-host="localhost:8086"'
(2)修改/usr/lib/systemd/system/kubelet.service
在ExecStart=xxx 后面加上${CADVISOR}
(3)重启kubelet
systemctl daemon-reload
systemctl restart kubelet.service
(4)访问http://192.168.1.12:4194/
12 K8S-WEB工具kubernetes-dashboard
kubernetes-dashboard 是一个WEB工具,可以对kubernates资源(POD、SERVICE、RC)进行增删改查:
增:新建(分填写的和上传json或yaml的),上传的方式,等同于kubectl create -f
删:删除副本(replicationcontrollers)
改:修改副本数
查:查询相关信息,同kubectl get命令
MASTER要安装flannel,请参数“FLANNEL安装与配置”章节
从kubernetes.tar.gz解压目录的如下路径拷贝YAML文件:
/root/kubernetes/cluster/addons/dashboard/dashboard-controller.yaml
/root/kubernetes/cluster/addons/dashboard/dashboard-service.yaml
修改dashboard-controller.yaml ,增加两行
ports:
- containerPort: 9090
args: (加上这两行)
- --apiserver-host=http://192.168.1.11:8080(加上这两行)
kubectl create -f dashboard-controller.yaml
kubectl create -f dashboard-service.yaml
http://192.168.1.11:8080/ui (192.168.1.11是Master节点)
13 集群监控Heapster+Influxdb+grafana
Heapster当前版本需使用https安全方式与master连接,
13.1 master主机创建证书并应用
mkdir key
cd key/
openssl genrsa -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key -subj "/CN=master.k8s.com" -days 5000 -out ca.crt
openssl genrsa -out server.key 2048
more /etc/hostname
docker1
openssl req -new -key server.key -subj "/CN=docker1" -out server.csr
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 5000
[root@k8s_master key]# ll
total 24
-rw-r--r-- 1 root root 1107 Apr 17 17:25 ca.crt
-rw-r--r-- 1 root root 1679 Apr 17 17:23 ca.key
-rw-r--r-- 1 root root 17 Apr 17 17:29 ca.srl
-rw-r--r-- 1 root root 985 Apr 17 17:29 server.crt
-rw-r--r-- 1 root root 891 Apr 17 17:27 server.csr
-rw-r--r-- 1 root root 1675 Apr 17 17:26 server.key
生成6个文件,复制至/var/run/kubernetes/
[root@k8s_master key]# cp *.* /var/run/kubernetes/
[root@k8s_master key]# ll /var/run/kubernetes/
total 32
-rw-r--r-- 1 kube kube 1200 Apr 17 11:43 apiserver.crt
-rw------- 1 kube kube 1675 Apr 17 11:43 apiserver.key
-rw-r--r-- 1 root root 1107 Apr 17 17:39 ca.crt
-rw-r--r-- 1 root root 1679 Apr 17 17:39 ca.key
-rw-r--r-- 1 root root 17 Apr 17 17:39 ca.srl
-rw-r--r-- 1 root root 985 Apr 17 17:39 server.crt
-rw-r--r-- 1 root root 891 Apr 17 17:39 server.csr
-rw-r--r-- 1 root root 1675 Apr 17 17:39 server.key
修改apiserer的配置文件
vi apiserver
.....
# default admission control policies
KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota"
# Add your own!
KUBE_API_ARGS="--client-ca-file=/var/run/kubernetes/ca.crt \
--tls-private-key-file=/var/run/kubernetes/server.key \
--tls-cert-file=/var/run/kubernetes/server.crt"
说明:
--client-ca-file:根证书文件
--tls-cert-file:服务端证书文件
--tls-private-key-file:服务端私钥文件
重启apiserver服务
[root@k8s_master kubernetes]# systemctl restart kube-apiserver
[root@k8s_master kubernetes]# systemctl status kube-apiserver
修改controller-manager服务参数
vi controller-manager
# Add your own!
KUBE_CONTROLLER_MANAGER_ARGS="--service-account-private-key-file=/var/run/kubernetes/apiserver.key \
--root-ca-file=/var/run/kubernetes/ca.crt"
重启controller-manager服务
systemctl restart kube-controller-manager
systemctl status kube-controller-manager
kubectl get serviceaccounts --all-namespaces
kubectl get secrets --all-namespaces
kubectl describe secret default-token-2d412
查看容器证书:
kubectl exec -ti frontend-z7021 ls /var/run/secrets/kubernetes.io/serviceaccount
13.2 部署Influxdb+grafana
我们使用已搭建好的环境(192.168.1.11)
登陆192.168.1.11,新建数据库k8s
实施步骤:
13.2.1 部署influxdb
cat <<eof |="" sudo="" tee="" etc="" yum.repos.d="" influxdb.repo
[influxdb]
name = InfluxDB Repository - RHEL \$releasever
baseurl = https://repos.influxdata.com/rhel/\$releasever/\$basearch/stable
enabled = 1
gpgcheck = 1
gpgkey = https://repos.influxdata.com/influxdb.key
EOF
yum install -y influxdb
systemctl enable influxdb
systemctl start influxdb
systemctl status influxdb
查看配置文件:
rpm -qc influxdb
vi /etc/influxdb/influxdb.conf
hostname = "192.168.1.11"
查看influxdb启动端口(8091,8083,8086,8088)
netstat -nltp
进入influx命令行
influx
SHOW DATABASES
CREATE DATABASE k8sdb
use k8sdb
CREATE USER "root" WITH PASSWORD 'root123' WITH ALL PRIVILEGES
show user
INSERT cpu,host=test,region=us_west value=0.64
SELECT * FROM /.*/ LIMIT 1
13.2.2 部署grafana
http://docs.grafana.org/datasources/influxdb/
cd /usr/src
grafana-2.6.0 不支持influx-0.13,需要 grafana-3.0以上
(Because of the changes to the SHOW SERIES and SHOW TAG VALUES formats in InfluxDB 0.11, InfluxDB 0.13 will not work with the Query Editor in Grafana 2.6.)
yum install https://grafanarel.s3.amazonaws.com/builds/grafana-3.1.1-1470047149.x86_64.rpm
查看配置文件
rpm -qc grafana
systemctl enable grafana-server
systemctl start grafana-server
systemctl status grafana-server
查看启动端口(3000)
netstat -nltp
浏览器打开,
http://192.168.1.11:3000
默认admin/admin
添加data source:填写influxdb版本,ip,端口,用户名密码;测试数据库名,用户名密码;并进行连接测试。
参考文档:http://docs.grafana.org/datasources/influxdb/
--------------------
Name :k8sdb Default
Type :InfluxDB 0.9x
Http settings
Url :http://192.168.1.11:8086
Access :proxy
Http Auth: Basic Auth
User :admin
Password: admin
InfluxDB Details
Database :k8s
User :root
Password :root
新建bashborad,并保存(根据过滤条件可以查到之前添加的一套数据)
定义要监控的信息,首先点击grafana的UI界面的左边的Dashboards按钮,
然后点击New按钮,此时将会在Dashboards的旁边出现一个绿色图标,
移动鼠标到该图标上,将会有一系列表展现出来,这时选择Add Panel然后在选择Graph。
Dashboards-->new-->add panel -->graph --->panel data source --> select
13.3 配置heapster容器
vi heapster-controller.yaml
apiVersion: v1
kind: ReplicationController
metadata:
name: heapster
labels:
name: heapster
spec:
replicas: 1
selector:
name: heapster
template:
metadata:
labels:
name: heapster
spec:
containers:
- name: heapster
image: gcr.io/google_containers/heapster:v1.1.0
command:
- /heapster
- --source=kubernetes:http://192.168.1.11:8080?inClusterConfig=false&kubeletHttps=true&kubeletPort=10250&useServiceAccount=true&auth=
- --sink=influxdb:http://192.168.1.11:8086
创建pod
kubectl create -f heapster-controller.yaml
kubectl get pods
kubectl logs heapster-g5ik1
14 附录1
14.1 Flannel 参数(所有节点)
/opt/kubernetes/bin/flanneld --ip-masq -etcd-endpoints=http://192.168.1.11:4001 -etcd-prefix=/coreos.com/network
14.2 kube-apiserve参数(MASTER节点)
/opt/kubernetes/bin/kube-apiserver --logtostderr=true --v=4 --etcd-servers=http://192.168.1.11:4001 --insecure-bind-address=0.0.0.0 --insecure-port=8080 --kubelet-port=10250 --advertise-address=192.168.1.11 --allow-privileged=false --service-cluster-ip-range=10.10.10.0/24 --admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota --client-ca-file=/var/run/kubernetes/ca.crt --tls-private-key-file=/var/run/kubernetes/server.key --tls-cert-file=/var/run/kubernetes/server.crt
14.3 kube-controller-manager参数(MASTER节点)
/opt/kubernetes/bin/kube-controller-manager --logtostderr=true --v=4 --master=192.168.1.11:8080 --service-account-private-key-file=/var/run/kubernetes/apiserver.key
14.4 kube-scheduler 参数(MASTER节点)
/opt/kubernetes/bin/kube-scheduler --logtostderr=true --v=4 --master=192.168.1.11:8080 --log-dir=/tmp/kube-scheduler.log
14.5 kube-dns参数(MASTER节点)
/opt/kubernetes/bin/kube-dns --dns-port=53 --domain=bstk8s.com --kube-master-url=http://192.168.1.11:8080
14.6 kube-proxy 参数(NODE节点)
/opt/kubernetes/bin/kube-proxy --logtostderr=true --v=4 --hostname-override=192.168.1.12 --master=http://192.168.1.11:8080
14.7 Kubelet 参数(NODE节点)
/opt/kubernetes/bin/kubelet --logtostderr=true --v=4 --address=192.168.1.12 --port=10250 --hostname-override=192.168.1.12 --api-servers=192.168.1.11:8080 --allow-privileged=false --cluster_dns=192.168.1.11 --cluster_domain=bstk8s.com --cadvisor-port=4194 --storage-driver-db="cadvisor" --storage-driver-host="localhost:8086"
15 附录2
相关部署脚本和YAML文件
来自 “ ITPUB博客 ” ,链接:http://blog.itpub.net/10995764/viewspace-2124022/,如需转载,请注明出处,否则将追究法律责任。
转载于:http://blog.itpub.net/10995764/viewspace-2124022/