文章目录
入门实战-namespace
一、Namespace
（1）、namespace概述
- name是Kubernetes系统中非常重要的组成部分，它的主要作用就是用来实现多种不同的租户资源进行隔离
- 默认情况下，Kubernetes集群中所有的pod都是可以互相访问的，但实际情况下，需要对两个不同的pod进行隔离，这个时候就将两个pod划分到不同的namespace下，Kubernetes通过将集群内部的资源分配到不同的namespace中，可以形成逻辑上的组，乙方宝不同组的资源进行隔离使用和管理。
- 可以通过kubernetes的授权机制，将不同的namespace交给不同租户进行管理，这样就实现了多租户的资源隔离。此时还能结合kubernetes的资源配额机制，限定不同租户能占用的资源，例如CPU使用星、内存使用呈
等，来实现租户可用资源的管理。
二、查看namespace
- 在Kubernetes集群启动之后，系统会默认创建几个namespace
[root@master ~]# kubectl get namespaces
NAME STATUS AGE
default Active 4d3h # 所有未指定Namespace的对 象都会被分配在default命名空间
dev Active 60m # 之前手动创建的
kube-node-lease Active 4d3h # 查看个集群之间的状态又称心跳维护
kube-public Active 4d3h # 此命名空间下的资源可以被所有人访问（包括未认证用户）
kube-system Active 4d3h # 所有由Kubernetes系统创建的资源都处于这个命名空间
kubernetes-dashboard Active 3d1h # 部署图形化管理界面生成的namespace
- 、查看namespace下的kube-system的集群组件
[root@master ~]# kubectl get pods -n kube-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coredns-bccdc95cf-hf8fx 1/1 Running 4 4d3h 10.244.0.29 master <none> <none>
coredns-bccdc95cf-sqpkz 1/1 Running 3 4d3h 10.244.0.30 master <none> <none>
etcd-master 1/1 Running 2 4d3h 10.0.0.10 master <none> <none>
kube-apiserver-master 1/1 Running 2 4d3h 10.0.0.10 master <none> <none>
kube-controller-manager-master 1/1 Running 2 4d3h 10.0.0.10 master <none> <none>
kube-flannel-ds-8jpqb 1/1 Running 3 4d3h 10.0.0.10 master <none> <none>
kube-flannel-ds-qm56c 1/1 Running 0 4d3h 10.0.0.20 worker1 <none> <none>
kube-flannel-ds-t7nts 1/1 Running 0 4d3h 10.0.0.30 worker2 <none> <none>
kube-proxy-dnpxl 1/1 Running 0 4d3h 10.0.0.20 worker1 <none> <none>
kube-proxy-r7szb 1/1 Running 2 4d3h 10.0.0.10 master <none> <none>
kube-proxy-stbzr 1/1 Running 0 4d3h 10.0.0.30 worker2 <none> <none>
kube-scheduler-master 1/1 Running 2 4d3h 10.0.0.10 master <none> <none>
- 查看所有的namespace
[root@master ~]# kubectl get ns
NAME STATUS AGE
default Active 4d4h
dev Active 70m
kube-node-lease Active 4d4h
kube-public Active 4d4h
kube-system Active 4d4h
kubernetes-dashboard Active 3d2h
- 查看指定的namespace
[root@master ~]# kubectl get namespaces dev
NAME STATUS AGE
dev Active 70m
- 查看namespace test输出指定的格式
[root@master ~]# kubectl get namespaces dev -o yaml
apiVersion: v1
kind: Namespace
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"Namespace","metadata":{"annotations":{},"name":"dev"}}
creationTimestamp: "2022-05-10T14:16:03Z"
name: dev
resourceVersion: "24807"
selfLink: /api/v1/namespaces/dev
uid: 6594adea-6cd3-432c-a034-0745e6429222
spec:
finalizers:
- kubernetes
status:
phase: Active
- 查看namespace test的描述信息
[root@master ~]# kubectl describe ns kubernetes-dashboard
Name: kubernetes-dashboard
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"v1","kind":"Namespace","metadata":{"annotations":{},"name":"kubernetes-dashboard"}}
Status: Active ==>Active状态表示正在运行，Terminating表示正在删除
No resource quota.
No resource limits.
[root@master ~]# kubectl describe ns dev
Name: dev
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"v1","kind":"Namespace","metadata":{"annotations":{},"name":"dev"}}
Status: Active
# ResourceQuota针对namespace做的资源限制
# LimitRange针对namespace中的每个组件做的资源限制
No resource quota.
No resource limits.
三、创建和删除namespace test
- 创建
[root@master ~]# kubectl create ns power
namespace/power created
#查看状态
[root@master ~]# kubectl get ns power
NAME STATUS AGE
power Active 8s
- 删除
[root@master ~]# kubectl delete ns power
namespace "power" deleted
#验证是否删除
[root@master ~]# kubectl get ns power
Error from server (NotFound): namespaces "power" not found
- 使用配置文件创建一个namespace test
[root@master ~]# vim ns-power.yaml
apiVersion: v1
kind: Namespace
metadata:
name: power
#创建namespace test
[root@master ~]# kubectl create -f ns-power.yaml
namespace/power created
#查看状态
[root@master ~]# kubectl get ns power
NAME STATUS AGE
power Active 7s
#删除该namespace test
[root@master ~]# kubectl delete -f ns-power.yaml
namespace "power" deleted
#验证是否删除
[root@master ~]# kubectl get ns power
Error from server (NotFound): namespaces "power" not found
四、Pod
Pod概述
- Pod是Kubernetes集群进行管理的最小单元，程序的运行和环境需要部署在容器中，而容器必须存在于Pod中
- Pod可以视作为容器的封装，一个Pod可以存在一个或者多个容器
（1）、查看容器
- 查看Pod容器
[root@master ~]# kubectl get pods --namespace kube-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coredns-bccdc95cf-hf8fx 1/1 Running 4 4d4h 10.244.0.29 master <none> <none>
coredns-bccdc95cf-sqpkz 1/1 Running 3 4d4h 10.244.0.30 master <none> <none>
etcd-master 1/1 Running 2 4d4h 10.0.0.10 master <none> <none>
kube-apiserver-master 1/1 Running 2 4d4h 10.0.0.10 master <none> <none>
kube-controller-manager-master 1/1 Running 2 4d4h 10.0.0.10 master <none> <none>
kube-flannel-ds-8jpqb 1/1 Running 3 4d4h 10.0.0.10 master <none> <none>
kube-flannel-ds-qm56c 1/1 Running 0 4d4h 10.0.0.20 worker1 <none> <none>
kube-flannel-ds-t7nts 1/1 Running 0 4d4h 10.0.0.30 worker2 <none> <none>
kube-proxy-dnpxl 1/1 Running 0 4d4h 10.0.0.20 worker1 <none> <none>
kube-proxy-r7szb 1/1 Running 2 4d4h 10.0.0.10 master <none> <none>
kube-proxy-stbzr 1/1 Running 0 4d4h 10.0.0.30 worker2 <none> <none>
kube-scheduler-master 1/1 Running 2 4d4h 10.0.0.10 master <none> <none>
（2）、创建Nginx Pod
- 创建一个Nginx的pod
# 命令格式： kubectl run (pod控制器名称) [参数]
# --image 指定Pod的镜像
# --port 指定端口
# --namespace 指定namespace
#创建一个Nginx的pod
[root@master ~]# kubectl run nginx --image=nginx:latest --port=80 --namespace dev
kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
deployment.apps/nginx created
#查看是否运行
[root@master ~]# kubectl get pod -n dev
NAME READY STATUS RESTARTS AGE
nginx-5ff9d6cc77-xt68z 1/1 Running 0 2m22s
#后面追加参数“-o wide”查看更为详细的参数
[root@master ~]# kubectl get pod -n dev -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-5ff9d6cc77-xt68z 1/1 Running 0 3m23s 10.244.0.36 master <none> <none>
#查看Nginx pod的详细描述信息
[root@master ~]# kubectl describe pod nginx-5ff9d6cc77-xt68z -n dev
Name: nginx-5ff9d6cc77-xt68z
Namespace: dev
Priority: 0
Node: master/10.0.0.10
Start Time: Wed, 11 May 2022 00:14:04 +0800
Labels: pod-template-hash=5ff9d6cc77
run=nginx
Annotations: <none>
Status: Running
IP: 10.244.0.36
Controlled By: ReplicaSet/nginx-5ff9d6cc77
Containers:
nginx:
Container ID: docker://a7c46e132845d6e335f572a8010dd5ec44f82d1d46e113c246bcabc631cf3c86
Image: nginx:latest
Image ID: docker-pullable://nginx@sha256:0d17b565c37bcbd895e9d92315a05c1c3c9a29f762b011a10c54a66cd53c9b31
Port: 80/TCP
Host Port: 0/TCP
State: Running
Started: Wed, 11 May 2022 00:14:20 +0800
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-mpqp8 (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
default-token-mpqp8:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-mpqp8
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 9m33s default-scheduler Successfully assigned dev/nginx-5ff9d6cc77-xt68z to master
Normal Pulling 9m32s kubelet, master Pulling image "nginx:latest"
Normal Pulled 9m17s kubelet, master Successfully pulled image "nginx:latest"
Normal Created 9m17s kubelet, master Created container nginx
Normal Started 9m17s kubelet, master Started container nginx
（3）、查看Nginx Pod的ip和映射的端口，访问
- 查看端口
[root@master ~]# kubectl get pods -n dev -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-5ff9d6cc77-xt68z 1/1 Running 0 13m 10.244.0.36 master <none> <none>
- 验证
[root@master ~]# curl 10.244.0.36:80
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
（4）、删除指定Pod
- Kubernetes管理pod是采用Pod控制器管理的，当前Pod是由Pod控制器创建的，控制器会监控Pod状况，一旦发现Pod死亡，会立即重建
- 如果想要删除Pod，需要删除Pod控制器
#查看Pod此时的状态
[root@master ~]# kubectl get pod -n dev
NAME READY STATUS RESTARTS AGE
nginx-5ff9d6cc77-xt68z 1/1 Running 0 22m
#查看Pod控制器
[root@master ~]# kubectl get deploy -n dev
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 1/1 1 1 23m
#删除控制器
[root@master ~]# kubectl delete deployment nginx -n dev
deployment.extensions "nginx" deleted
#验证是否删除
[root@master ~]# kubectl get deploy -n dev
No resources found.
[root@master ~]# kubectl get pod -n dev
No resources found.
（5）、yaml语言介绍
Yaml概述:
YAML是一个类似于XML、JSON的标记性语言,它强调以数据为中心,并不是以标识语言为重点.因而YAML本身的定义比较简单,号称一种人性化的数据格式语言.
Yaml语法:
①.大小写敏感
②.使用缩进表示层级关系
③.缩进不允许使用Tab,只允许使用空格(对于低版本的限制)
④.缩进的空格数不重要,只要相同层级的元素左对齐即可
⑤.'#'表示注释
- yaml支持以下几种数据类型
| 数据类型 |
|---|
| 纯量：单个的、不可再分的值 |
| 对象：键值对的集合，又称为映射（mapping）/ 哈希（hash） / 字典（dictionary） |
| 数组：一组按次序排列的值，又称为序列（sequence） / 列表（list） |
# 纯量, 就是指的一个简单的值，字符串、布尔值、整数、浮点数、Null、时间、日期
# 1 布尔类型
c1: true (或者True)
# 2 整型
c2: 234
# 3 浮点型
c3: 3.14
# 4 null类型
c4: ~ # 使用~表示null
# 5 日期类型
c5: 2018-02-17 # 日期必须使用ISO 8601格式，即yyyy-MM-dd
# 6 时间类型
c6: 2018-02-17T15:02:31+08:00 # 时间使用ISO 8601格式，时间和日期之间使用T连接，最后使用+代表时区
# 7 字符串类型
c7: heima # 简单写法，直接写值 , 如果字符串中间有特殊字符，必须使用双引号或者单引号包裹
c8: line1
line2 # 字符串过多的情况可以拆成多行，每一行会被转化成一个空格
# 对象
# 形式一(推荐):
heima:
age: 15
address: Beijing
# 形式二(了解):
heima: {age: 15,address: Beijing}
Tips：
1 书写yaml切记
:后面要加一个空格2 如果需要将多段yaml配置放在一个文件中，中间要使用
---分隔3 下面是一个yaml转json的网站，可以通过它验证yaml是否书写正确
https://www.json2yaml.com/convert-yaml-to-json
（6）、使用yaml配置文件创建一一个Nginx Pod
- 首先编写一个Nginx的yaml文件
[root@master ~]# vim pod-nginx.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx
namespace: dev
spec:
containers:
- image: nginx:latest
name: pod
ports:
- name: nginx-port
containerPort: 80
protocol: TCP
- 创建
[root@master ~]# kubectl create -f pod-nginx.yaml
pod/nginx created
#查看状态
[root@master ~]# kubectl get pod -n dev
NAME READY STATUS RESTARTS AGE
nginx 1/1 Running 0 44s
- 删除Nginx Pod
[root@master ~]# kubectl delete -f pod-nginx.yaml
pod "nginx" deleted
#查看状态
[root@master ~]# kubectl get pod -n dev
No resources found.
四、多种配置方式
- （1）、命令式对象配置：
命令式对象配置就是使用命令配合配置文件一起来操作kubernetes资源
创建一个nginxpod.yaml:输入以下内容
[root@zabbix-server ~]# vim nginxpod.yaml
apiVersion: v1
kind: Namespace
metadata:
name: pods
---
apiVersion: v1
kind: Pod
metadata:
name: nginxpod
namespace: pods
spec:
containers:
- name: nginx-conntainers
image: nginx:1.20.0
- 执行create命令,创建一个资源
[root@zabbix-server ~]# kubectl create -f nginxpod.yaml
namespace/dev created
pod/nginxpod created
[root@k8s-master ~]# kubectl get pod -n pods
NAME READY STATUS RESTARTS AGE
nginxpod 1/1 Running 0 117s
#查看Nginx容器的详细数据
[root@k8s-master ~]# kubectl describe pod -n pods nginxpod
Name: nginxpod
Namespace: pods
Priority: 0
Node: computer/10.0.0.10
Start Time: Wed, 10 Aug 2022 14:52:36 +0800
Labels: <none>
Annotations: <none>
Status: Running
IP: 10.244.1.7
IPs:
IP: 10.244.1.7
Containers:
nginx-conntainers:
Container ID: docker://c4d1afe15ddcfeb7d795fac337e1f5e3d3cdf1f5af77528fe8096ada9b47f48a
Image: nginx:1.20.0
Image ID: docker-pullable://nginx@sha256:ea4560b87ff03479670d15df426f7d02e30cb6340dcd3004cdfc048d6a1d54b4
Port: <none>
Host Port: <none>
State: Running
Started: Wed, 10 Aug 2022 14:53:24 +0800
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-jl2jr (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
kube-api-access-jl2jr:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 112s default-scheduler Successfully assigned pods/nginxpod to computer
Normal Pulling 111s kubelet Pulling image "nginx:1.20.0"
Normal Pulled 65s kubelet Successfully pulled image "nginx:1.20.0" in 46.413550509s
Normal Created 64s kubelet Created container nginx-conntainers
Normal Started 64s kubelet Started container nginx-conntainers
#查看IP地址
[root@k8s-master ~]# kubectl get pod -n pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginxpod 1/1 Running 0 5m18s 10.244.1.7 computer <none> <none>
[root@k8s-master ~]# curl 10.244.1.7
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
- （2）、声明式对象配置
声明式对象配置: 通过apply命令和配置文件去操作kubernetes资源,apply命令和create的区别就是,如果yaml文件里面的环境已有就是更新,没有就是创建.
#我们修改一下刚刚的nginxpod.yaml文件
[root@k8s-master ~]# cat nginxpod.yaml
apiVersion: v1
kind: Namespace
metadata:
name: pods
---
apiVersion: v1
kind: Pod
metadata:
name: nginxpod
namespace: pods
spec:
containers:
- name: nginx-conntainers
image: nginx:1.21.0 ==>修改为1.21.0
#执行apply命令
[root@k8s-master ~]# kubectl apply -f nginxpod.yaml
namespace/pods configured
pod/nginxpod configured
#重新查看详细信息
[root@k8s-master ~]# kubectl describe pod -n pods nginxpod
Name: nginxpod
Namespace: pods
Priority: 0
Node: computer/10.0.0.10
Start Time: Wed, 10 Aug 2022 14:52:36 +0800
Labels: <none>
Annotations: <none>
Status: Running
IP: 10.244.1.7
IPs:
IP: 10.244.1.7
Containers:
nginx-conntainers:
Container ID: docker://c4d1afe15ddcfeb7d795fac337e1f5e3d3cdf1f5af77528fe8096ada9b47f48a
Image: nginx:1.21.0
Image ID: docker-pullable://nginx@sha256:ea4560b87ff03479670d15df426f7d02e30cb6340dcd3004cdfc048d6a1d54b4
Port: <none>
Host Port: <none>
State: Running
Started: Wed, 10 Aug 2022 14:53:24 +0800
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-jl2jr (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
kube-api-access-jl2jr:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 11m default-scheduler Successfully assigned pods/nginxpod to computer
Normal Pulling 11m kubelet Pulling image "nginx:1.20.0"
Normal Pulled 10m kubelet Successfully pulled image "nginx:1.20.0" in 46.413550509s
Normal Created 10m kubelet Created container nginx-conntainers
Normal Started 10m kubelet Started container nginx-conntainers
Normal Killing 33s kubelet Container nginx-conntainers definition changed, will be restarted
Normal Pulling 33s kubelet Pulling image "nginx:1.21.0" ==>发现image version已经更新
在node节点上执行kubectl命令
#首先我们要把master主节点上的环境变量拷贝到node节点下
[root@k8s-master ~]# scp -r ~/.kube/ computer:~/
#或者设置一个环境变量也可以
| 创建更资源 | 使用声明式对象配置: kubectl apply -f xxx.taml |
|---|---|
| 删除资源 | 使用命令式对象配置: kubectl delete -f xxx.yaml |
| 查询资源 | 使用命令式对象管理: kubectl get (describe) 资源名称 |
五、Pod
Pod
Pod 是k8s集群进行管理的最小单元,程序要运行在部署的容器当中,而容器必须存在于 Pod
中,Pod 可以认为是容器的封装,一个Pod中可以存在一个或者多个容器.k8s多少个节点就有对应的服务
k8s对于Pod的管理是通过Pod控制器,如果当pod控制器发现一个pod死亡,会立即重建一个pod,为pod提供高可用、冗余服务.
所以想要删除一个pod容器必须删除pod控制器.
| 容器 | 功能 |
|---|---|
| ①.coredns: | 用于k8s的dns解析 |
| ②.etcd: | 负责存储k8s整个集群的状态 |
| ③.apiserver: | 负责消息的接收 |
| ④.controller-manager: | 负责进行提供服务 |
| ⑤.flannel: | 负责k8s集群中的网络 |
[root@xnode1 ~]# kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-8686dcc4fd-f9pqv 1/1 Running 2 5h57m
coredns-8686dcc4fd-mtcmn 1/1 Running 5 5h57m
etcd-xnode1 1/1 Running 2 5h56m
kube-apiserver-xnode1 1/1 Running 2 5h56m
kube-controller-manager-xnode1 1/1 Running 2 5h56m
kube-flannel-ds-amd64-67htn 1/1 Running 2 5h41m
kube-flannel-ds-amd64-frh8l 1/1 Running 2 5h45m
kube-proxy-ptlm8 1/1 Running 2 82m
kube-proxy-rjgfs 1/1 Running 0 82m
kube-scheduler-xnode1 1/1 Running 3 5h56m
#创建一个容器
[root@xnode1 ~]# kubectl create namespace dev
namespace/dev created
[root@xnode1 ~]# kubectl run nginx --image=nginx:1.17.1 --port=80 --namespace=dev
kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
deployment.apps/nginx created
#查看容器的详细信息
[root@xnode1 ~]# kubectl describe pod -n dev nginx-64959c9fb5-szlnj
Name: nginx-64959c9fb5-szlnj
Namespace: dev
Priority: 0
PriorityClassName: <none>
Node: xnode2/10.0.0.40
Start Time: Wed, 17 Aug 2022 05:04:40 -0400
Labels: pod-template-hash=64959c9fb5
run=nginx
Annotations: <none>
Status: Running
IP: 10.244.1.15
Controlled By: ReplicaSet/nginx-64959c9fb5
Containers:
nginx:
Container ID: docker://9d76ced9a4d34619d56ab1a5775432abf028d4cc19cc73115a36737021431d7c
Image: nginx:1.17.1
Image ID: docker-pullable://nginx@sha256:b4b9b3eee194703fc2fa8afa5b7510c77ae70cfba567af1376a573a967c03dbb
Port: 80/TCP
Host Port: 0/TCP
State: Running
Started: Wed, 17 Aug 2022 05:04:42 -0400
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-wltjd (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
default-token-wltjd:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-wltjd
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 2m2s default-scheduler Successfully assigned dev/nginx-64959c9fb5-szlnj to xnode2
Normal Pulled 2m kubelet, xnode2 Container image "nginx:1.17.1" already present on machine
Normal Created 2m kubelet, xnode2 Created container nginx
Normal Started 2m kubelet, xnode2 Started container nginx
#访问该Nginx pod:查询dev namespace下的Nginxpod容器IP地址
[root@xnode1 ~]# kubectl get pod -n dev nginx-64959c9fb5-szlnj -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-64959c9fb5-szlnj 1/1 Running 0 14m 10.244.1.15 xnode2 <none> <none>
[root@xnode1 ~]# curl 10.244.1.15:80
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
查看pod控制器
删除pod:先将pod控制器删除,pod也就没了
[root@xnode1 ~]# kubectl get deployment -n dev
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 1/1 1 1 20m
[root@xnode1 ~]# kubectl delete deployment nginx -n dev
deployment.extensions "nginx" deleted
[root@xnode1 ~]# kubectl get pod -n dev
No resources found.
六、Label
Label:
Label是k8s系统中的一个重要的概念,它的作用就是在资源商添加标识,对他们进行区分和选择.
Label 会在key/value键值对的形式附加到各种对象上,如 Pod 、Node 、Service
一个资源对象可以定义任意数量的 Label，同一个 Label也可以被添加到任意数量的资源对象上去
Label可以通过对象式配置时添加到yaml配置文件里面
| 分类 | 标签 |
|---|---|
| 版本标签: | version: xxx |
| 环境标签: | environment:xxx |
| 架构标签: | tier:xxx |
- (1)、label用于给某个资源对象定义标识而label selector用于查看和筛选拥有某些标签的资源对象
基于等式的Label Selector
name = slave: 选择所有包含Label中key="name"且value="slave"的对象
env != production: 选择所有包括Label中的key="env"且value不等于"production"的对象
基于集合的Label Selector
name in (master, slave): 选择所有包含Label中的key="name"且value="master"或"slave"的对象
name not in (frontend): 选择所有包含Label中的key="name"且value不等于"frontend"的对象
标签的选择条件可以使用多个，此时将多个Label Selector进行组合，使用逗号","进行分隔即可。例如：
name=slave，env!=production
name not in (frontend)，env!=production
#查看资源标签
[root@xnode1 ~]# kubectl get pod -n dev
NAME READY STATUS RESTARTS AGE
nginx 0/1 ContainerCreating 0 6s
[root@xnode1 ~]# kubectl get pod -n dev --show-labels
NAME READY STATUS RESTARTS AGE LABELS
nginx 0/1 ContainerCreating 0 13s <none>
#给Nginx这个资源打上标签
[root@xnode1 ~]# kubectl label pod nginx -n dev version=1.0
pod/nginx labeled
[root@xnode1 ~]# kubectl get pod -n dev --show-labels
NAME READY STATUS RESTARTS AGE LABELS
nginx 1/1 Running 0 91s version=1.0
#追加更新标签:
[root@xnode1 ~]# kubectl label pod nginx -n dev tier=back
pod/nginx labeled
[root@xnode1 ~]# kubectl get pod -n dev --show-labels
NAME READY STATUS RESTARTS AGE LABELS
nginx 1/1 Running 0 3m12s tier=back,version=1.0
#覆盖更新标签:
[root@xnode1 ~]# kubectl label pod nginx -n dev --overwrite version=2.0
pod/nginx labeled
[root@xnode1 ~]# kubectl get pod -n dev --show-labels
NAME READY STATUS RESTARTS AGE LABELS
nginx 1/1 Running 0 4m16s tier=back,version=2.0
#筛选标签
[root@xnode1 ~]# kubectl create -f nginx.yaml
pod/nginx01 created
[root@xnode1 ~]# kubectl label pod nginx01 -n dev version=1.0
pod/nginx01 labeled
[root@xnode1 ~]# kubectl get pod -n dev nginx01 --show-labels
NAME READY STATUS RESTARTS AGE LABELS
nginx01 1/1 Running 0 86s version=1.0
#对着两个label的version进行筛选
[root@xnode1 ~]# kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
nginx 1/1 Running 0 8m44s
nginx01 1/1 Running 0 2m11s
[root@xnode1 ~]# kubectl get pods -n dev --show-labels
NAME READY STATUS RESTARTS AGE LABELS
nginx 1/1 Running 0 8m54s tier=back,version=2.0
nginx01 1/1 Running 0 2m21s version=1.0
#加上参数-l "筛选label名字"
[root@xnode1 ~]# kubectl get pod -l "version=1.0" -n dev --show-labels
NAME READY STATUS RESTARTS AGE LABELS
nginx01 1/1 Running 0 3m21s version=1.0
#删除标签
[root@xnode1 ~]# kubectl label pod nginx -n dev tier-
pod/nginx labeled
[root@xnode1 ~]# kubectl get pod -n dev
NAME READY STATUS RESTARTS AGE
nginx 1/1 Running 0 11m
nginx01 1/1 Running 0 4m39s
[root@xnode1 ~]# kubectl get pod -n dev --show-labels
NAME READY STATUS RESTARTS AGE LABELS
nginx 1/1 Running 0 11m version=2.0
nginx01 1/1 Running 0 5m3s version=1.0
对象配置方式设置label:
[root@xnode1 ~]# cat >>nginx.yaml<<EOF
apiVersion: v1
kind: Pod
metadata:
name: nginx
namespace: dev
labels:
version: "3.0"
env: "test"
spec:
containers:
- image: nginx:latest
name: pod
ports:
- name: nginx-port
containerPort: 80
protocol: TCP
EOF
[root@xnode1 ~]# kubectl create -f nginx.yaml
pod/nginx02 created
[root@xnode1 ~]# kubectl get pods -n dev --show-labels
NAME READY STATUS RESTARTS AGE LABELS
nginx 1/1 Running 0 16m version=2.0
nginx01 1/1 Running 0 9m58s version=1.0
nginx02 0/1 ContainerCreating 0 22s env=test,version=3.0
七、deployment
Deployment:
在kubernetes中,Pod是最少的控制单元,但是kubernetes很少直接控制Pod,一般pod是由Pod控制器管理,确保Pod处于预期的状态,当一个Pod出现故障时,会尝试重启或者进行自动修复或重建Pod
#使用命令创建一个Pod
[root@xnode1 ~]# kubectl run nginx --image=nginx:1.17.1 --port=80 --replicas=3 --namespace=dev
kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
deployment.apps/nginx created
[root@xnode1 ~]# kubectl get pod -n dev
NAME READY STATUS RESTARTS AGE
nginx-64959c9fb5-9556z 1/1 Running 0 11s
nginx-64959c9fb5-pzdtj 1/1 Running 0 11s
nginx-64959c9fb5-s4n5j 1/1 Running 0 11s
#并查看这个Pod控制器下的Pod
[root@xnode1 ~]# kubectl get deployment,pods -n dev
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.extensions/nginx 3/3 3 3 34s
NAME READY STATUS RESTARTS AGE
pod/nginx-64959c9fb5-9556z 1/1 Running 0 34s
pod/nginx-64959c9fb5-pzdtj 1/1 Running 0 34s
pod/nginx-64959c9fb5-s4n5j 1/1 Running 0 34s
#查看Pod控制器的详细信息
[root@xnode1 ~]# kubectl describe deployment nginx -n dev
Name: nginx
Namespace: dev
CreationTimestamp: Wed, 17 Aug 2022 21:09:41 -0400
Labels: run=nginx
Annotations: deployment.kubernetes.io/revision: 1
Selector: run=nginx
Replicas: 3 desired | 3 updated | 3 total | 3 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: run=nginx
Containers:
nginx:
Image: nginx:1.17.1
Port: 80/TCP
Host Port: 0/TCP
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
Progressing True NewReplicaSetAvailable
OldReplicaSets: <none>
NewReplicaSet: nginx-64959c9fb5 (3/3 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 2m51s deployment-controller Scaled up replica set nginx-64959c9fb5 to 3
[root@xnode1 ~]# kubectl get pods -n dev --show-labels
NAME READY STATUS RESTARTS AGE LABELS
nginx-64959c9fb5-9556z 1/1 Running 0 4m1s pod-template-hash=64959c9fb5,run=nginx
nginx-64959c9fb5-pzdtj 1/1 Running 0 4m1s pod-template-hash=64959c9fb5,run=nginx
nginx-64959c9fb5-s4n5j 1/1 Running 0 4m1s pod-template-hash=64959c9fb5,run=nginx
#删除deployment
[root@xnode1 ~]# kubectl delete deployment nginx -n dev
deployment.extensions "nginx" deleted
使用配置文件删除deployment
[root@xnode1 ~]# cat >>deploy-nginx.yaml<<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
run: nginx
template:
metadata:
labels:
run: nginx
spec:
containers:
- image: nginx:latest
name: nginx
ports:
- containerPort: 80
protocol: TCP
EOF
[root@xnode1 ~]# kubectl create -f deploy-nginx.yaml
deployment.apps/nginx created
[root@xnode1 ~]# kubectl get deployment,pods -n dev
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.extensions/nginx 2/3 3 2 23s
NAME READY STATUS RESTARTS AGE
pod/nginx-69578d4d9b-4jn8w 1/1 Running 0 23s
pod/nginx-69578d4d9b-7xx6g 0/1 ContainerCreating 0 23s
pod/nginx-69578d4d9b-n79rt 1/1 Running 0 23s
#尝试删除deployment
[root@xnode1 ~]# kubectl delete deployment -n dev nginx
deployment.extensions "nginx" deleted
[root@xnode1 ~]#
[root@xnode1 ~]# kubectl get deployment,pods -n dev
NAME READY STATUS RESTARTS AGE
pod/nginx-69578d4d9b-4jn8w 0/1 Terminating 0 93s
pod/nginx-69578d4d9b-7xx6g 0/1 Terminating 0 93s
pod/nginx-69578d4d9b-n79rt 0/1 Terminating 0 93s
八、Service
Pod在创建的时候会分配一个Ip
但是Pod的IP会随着的重建而产生变化
Pod的IP仅仅是集群内部的IP,外部无法访问
①.这个时候kubernetes设计了Services来解决这个问题,Services可以看做是同一组里面同类Pod 对外访问的一个IP,借助Service,应用可以方便的实现服务的发现和负载均衡.
②.只要不删除Service,那么这个ServiceIp是不变得
[root@xnode1 ~]# kubectl expose deployment nginx --name=svc-nginx1 --type=ClusterIP --port=80 --target-port=80 -n dev
service/svc-nginx1 exposed
[root@xnode1 ~]# kubectl get service -n dev
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
svc-nginx1 ClusterIP 10.100.168.159 <none> 80/TCP 28s
#访问以下ClusterIP
[root@xnode1 ~]# curl 10.100.168.159:80
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
上面创建的Service的Type类型为ClusterIP,这个IP地址只用集群内部可访问
如果需要创建外部也可以访问的Service,需要修改Type为NodePort
[root@xnode1 ~]# kubectl expose deployment nginx --name=svc-nginx2 --type=NodePort --port=80 --target-port=80 -n dev
service/svc-nginx2 exposed
#查看验证
[root@xnode1 ~]# kubectl get svc -n dev
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
svc-nginx1 ClusterIP 10.100.168.159 <none> 80/TCP 12m
svc-nginx2 NodePort 10.101.90.131 <none> 80:30413/TCP 24s ==>这个端口映射到master集群的端口
- 我们在这里访问一下 30413 这个端口
#删除Service
[root@xnode1 ~]# kubectl delete svc svc-nginx1 -n dev
service "svc-nginx1" deleted
[root@xnode1 ~]# kubectl get svc -n dev
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
svc-nginx2 NodePort 10.101.90.131 <none> 80:30413/TCP 6m18s
使用命令式配置对象的方式创建svc-nginx
[root@xnode1 ~]# cat <<EOF >svc-nginx.yaml
apiVersion: v1
kind: Service
metadata:
name: svc-nginx
namespace: dev
spec:
clusterIP: 10.109.179.231 #固定svc的内网ip
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: ClusterIP
EOF
[root@xnode1 ~]# kubectl create -f svc-nginx.yaml
service/svc-nginx created
[root@xnode1 ~]# kubectl get svc -n dev
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
svc-nginx ClusterIP 10.109.179.231 <none> 80/TCP 13s
svc-nginx2 NodePort 10.101.90.131 <none> 80:30413/TCP 11m
#创建并删除
[root@xnode1 ~]# kubectl delete -f svc-nginx.yaml
service "svc-nginx" deleted
[root@xnode1 ~]# kubectl get svc -n dev
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
svc-nginx2 NodePort 10.101.90.131 <none> 80:30413/TCP 12m
九、Pod详解
（1）、Pod概述
每个Pod中都可以包含一个或者多个容器,这些容器可以分为两类
用户所在的容器,容器数量可多可少
Pause 容器,这是每个Pod都会有的一个 根容器 ,它的作用有以下两个
①.可以以它为依据,评估整个Pod的健康状态
②.可以在根容器上设置IP地址,其它的容器都根据此Pod IP来实现Pod的内部通信
③.Pod内部的通信,Pod之间的通讯采用虚拟的二层网络技术实现的,我们当前使用的的flannel
#查看Pod的1级属性
[root@xnode1 ~]# kubectl explain pod
KIND: Pod
VERSION: v1
DESCRIPTION:
Pod is a collection of containers that can run on a host. This resource is
created by clients and scheduled onto hosts.
FIELDS:
apiVersion <string>
APIVersion defines the versioned schema of this representation of an
object. Servers should convert recognized schemas to the latest internal
value, and may reject unrecognized values. More info:
https://git.k8s.io/community/contributors/devel/api-conventions.md#resources
kind <string>
Kind is a string value representing the REST resource this object
represents. Servers may infer this from the endpoint the client submits
requests to. Cannot be updated. In CamelCase. More info:
https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds
metadata <Object>
Standard object's metadata. More info:
https://git.k8s.io/community/contributors/devel/api-conventions.md#metadata
spec <Object>
Specification of the desired behavior of the pod. More info:
https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status
status <Object>
Most recently observed status of the pod. This data may not be up to date.
Populated by the system. Read-only. More info:
https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status
#查看Pod的2级属性
[root@xnode1 ~]# kubectl explain pod.apiVersion
KIND: Pod
VERSION: v1
FIELD: apiVersion <string>
DESCRIPTION:
APIVersion defines the versioned schema of this representation of an
object. Servers should convert recognized schemas to the latest internal
value, and may reject unrecognized values. More info:
https://git.k8s.io/community/contributors/devel/api-conventions.md#resources
#查看apiversion所有的版本
[root@xnode1 ~]# kubectl api-versions
（2）、Pod配置
[root@xnode1 ~]# kubectl explain pod.spec.containers
KIND: Pod
VERSION: v1
RESOURCE: containers <[]Object> # 数组，代表可以有多个容器
FIELDS:
name <string> # 容器名称
image <string> # 容器需要的镜像地址
imagePullPolicy <string> # 镜像拉取策略
command <[]string> # 容器的启动命令列表，如不指定，使用打包时使用的启动命令
args <[]string> # 容器的启动命令需要的参数列表
env <[]Object> # 容器环境变量的配置
ports <[]Object> # 容器需要暴露的端口号列表
resources <Object> # 资源限制和资源请求的设置
基本配置:
创建Pod-base.yaml文件:
[root@xnode1 ~]# cat <<EOF >pod-base.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-base
namespace: dev
labels:
user: heima
spec:
containers:
- name: nginx
image: nginx:1.17.1
- name: busybox
image: busybox:1.30
EOF
[root@xnode1 ~]# kubectl create -f pod-base.yaml
pod/pod-base created
#查看验证
[root@xnode1 ~]# kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
pod-base 0/2 ContainerCreating 0 22s
[root@xnode1 ~]# kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
pod-base 1/2 CrashLoopBackOff 1 28s
[root@xnode1 ~]# kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
pod-base 1/2 Running 2 45s
#查看这个容器的详细信息
[root@xnode1 ~]# kubectl describe pods -n dev pod-base
Name: pod-base
Namespace: dev
Priority: 0
PriorityClassName: <none>
Node: xnode2/10.0.0.40
Start Time: Wed, 17 Aug 2022 23:24:59 -0400
Labels: user=heima
Annotations: <none>
Status: Running
IP: 10.244.1.37
Containers:
nginx:
Container ID: docker://17fbb7874027d0238562a5840a9f12fcf5537b492d52cfad71fe2f29835435ba
Image: nginx:1.17.1
Image ID: docker-pullable://nginx@sha256:b4b9b3eee194703fc2fa8afa5b7510c77ae70cfba567af1376a573a967c03dbb
Port: <none>
Host Port: <none>
State: Running
Started: Wed, 17 Aug 2022 23:25:01 -0400
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-wltjd (ro)
busybox:
Container ID: docker://41c5263c5d5c81ebe4c36b554f96774fe4c0f49d8b0cc82468f72ee50bba65c0
Image: busybox:1.30
Image ID: docker-pullable://busybox@sha256:4b6ad3a68d34da29bf7c8ccb5d355ba8b4babcad1f99798204e7abb43e54ee3d
Port: <none>
Host Port: <none>
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Completed
Exit Code: 0
Started: Wed, 17 Aug 2022 23:26:02 -0400
Finished: Wed, 17 Aug 2022 23:26:02 -0400
Ready: False
Restart Count: 3
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-wltjd (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
default-token-wltjd:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-wltjd
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 100s default-scheduler Successfully assigned dev/pod-base to xnode2
Normal Pulled <invalid> kubelet, xnode2 Container image "nginx:1.17.1" already present on machine
Normal Created <invalid> kubelet, xnode2 Created container nginx
Normal Started <invalid> kubelet, xnode2 Started container nginx
Normal Pulling <invalid> kubelet, xnode2 Pulling image "busybox:1.30"
Normal Pulled <invalid> kubelet, xnode2 Successfully pulled image "busybox:1.30"
Normal Created <invalid> (x4 over <invalid>) kubelet, xnode2 Created container busybox
Normal Started <invalid> (x4 over <invalid>) kubelet, xnode2 Started container busybox
Normal Pulled <invalid> (x3 over <invalid>) kubelet, xnode2 Container image "busybox:1.30" already present on machine
Warning BackOff <invalid> (x6 over <invalid>) kubelet, xnode2 Back-off restarting failed container
#我们可以这个容器被尝试重启4次,查看详情发现容器启动时报错
[root@xnode1 ~]# kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
pod-base 1/2 CrashLoopBackOff 4 2m48s
（3）、镜像拉取策略
镜像拉取策略
imagePullPolicy,用于设置镜像拉取策略,kubernetes支持配置三种拉取策略:
- Always: 总是从远程仓库拉取镜像(默认一直是远程下载)
- IfNotPresent: 本地有镜像则使用本地的镜像,本地没有则从远程仓库拉取镜像(本地有惊险就用本地,没有则远程下载)
- Never: 只使用本地镜像,从不去远程仓库拉取,本地没有就报错(一直使用本地镜像)
[root@xnode1 ~]# cat <<EOF >pod-imagepullpolicy.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-imagepullpolicy
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.2
imagePullPolicy: Never # 用于设置镜像拉取策略
- name: busybox
image: busybox:1.30
EOF
[root@xnode1 ~]# kubectl create -f pod-imagepullpolicy.yaml
pod/pod-imagepullpolicy created
[root@xnode1 ~]# kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
pod-imagepullpolicy 1/2 CrashLoopBackOff 1 6s
#查看详细信息
[root@xnode1 ~]# kubectl describe pod -n dev pod-imagepullpolicy
... ...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 21s default-scheduler Successfully assigned dev/pod-imagepullpolicy to xnode2
Normal Pulled <invalid> (x3 over <invalid>) kubelet, xnode2 Container image "busybox:1.30" already present on machine
Normal Created <invalid> (x3 over <invalid>) kubelet, xnode2 Created container busybox
Normal Started <invalid> (x3 over <invalid>) kubelet, xnode2 Started container busybox
Warning ErrImageNeverPull <invalid> (x6 over <invalid>) kubelet, xnode2 Container image "nginx:1.17.2" is not present with pull policy of Never
Warning Failed <invalid> (x6 over <invalid>) kubelet, xnode2 Error: ErrImageNeverPull
Warning BackOff <invalid> (x3 over <invalid>) kubelet, xnode2 Back-off restarting failed container
查看本地Pod拉取镜像策略
[root@xnode1 ~]# kubectl explain pod.spec.containers
... ...
imagePullPolicy <string>
Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always
if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated.
More info:
https://kubernetes.io/docs/concepts/containers/images#updating-images
我们修改一下拉取镜像启动Pod的配置文件:
将原本空的镜像策略Never修改成IdNotPresent
[root@xnode1 ~]# cat <<EOF >pod-imagepullpolicy.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-imagepullpolicy
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.2
imagePullPolicy: IfNotPresent # 用于设置镜像拉取策略
- name: busybox
image: busybox:1.30
EOF
#重新创建
[root@xnode1 ~]# kubectl create -f pod-imagepullpolicy.yaml
pod/pod-imagepullpolicy created
#查看详细信息
[root@xnode1 ~]# kubectl describe pod -n dev pod-imagepullpolicy
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 55s default-scheduler Successfully assigned dev/pod-imagepullpolicy to xnode2
Normal Pulling <invalid> kubelet, xnode2 Pulling image "nginx:1.17.2"
Normal Pulled <invalid> kubelet, xnode2 Successfully pulled image "nginx:1.17.2"
Normal Created <invalid> kubelet, xnode2 Created container nginx
Normal Started <invalid> kubelet, xnode2 Started container nginx
Normal Pulled <invalid> (x2 over <invalid>) kubelet, xnode2 Container image "busybox:1.30" already present on machine
Normal Created <invalid> (x2 over <invalid>) kubelet, xnode2 Created container busybox
Normal Started <invalid> (x2 over <invalid>) kubelet, xnode2 Started container busybox
Warning BackOff <invalid> (x2 over <invalid>) kubelet, xnode2 Back-off restarting failed container
#查看docker images镜像
[root@xnode2 ~]# docker images | grep nginx
nginx latest 605c77e624dd 7 months ago 141MB
nginx 1.20.0 7ab27dbbfbdf 15 months ago 133MB
nginx 1.17.2 4733136e5c3c 3 years ago 126MB
nginx 1.17.1 98ebf73aba75 3 years ago 109MB
（4）、启动命令：command
在前面启动Pod中,一直有一个问题没有解决,就是busybox容器没有成功运行,是什么原因导致这个容器故障呢?
busybox并不是一个程序,而是一个类似于工具类的集合,kubernetes集群启动管理后,它就会自动关闭,解决方案就是让其一直在允许,这里就用到了command配置
command 就是k8s Pod运行起来了初始化执行的一个命令
对下面yaml文件里的 command 进行解释:
- “/bin/bash”,"-c"是一个shell命令解释器,表示执行sh命令
- touch /tmp/hello.txt/:在/tmp/目录下创建一个hello的txt文件
- while true;do /bin/echo $(date +%T) >> /tmp/hello.txt;sleep 3;done:每隔3秒向文件中写入当前时间.
[root@xnode1 ~]# cat <<EOF >pod-command.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-command
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
- name: busybox
image: busybox:1.30
command: ["/bin/sh","-c","touch /tmp/hello.txt;while true;do /bin/echo $(date +%T) >> /tmp/hello.txt; sleep 3; done;"]
EOF
#创建并查看验证
[root@xnode1 ~]# kubectl create -f pod-command.yaml
pod/pod-command created
[root@xnode1 ~]# kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
pod-command 2/2 Running 0 7s
pod-imagepullpolicy 1/2 CrashLoopBackOff 17 21h
#进入pod中的busybox容器查看文件内容
[root@xnode1 ~]# kubectl exec pod-command -n dev -it -c busybox /bin/sh
/ # tail -f /tmp/hello.txt
03:19:50
03:19:50
03:19:50
03:19:50
（5）、环境变量
- 创建一个pod-env.yaml文件
[root@xnode1 ~]# cat <<EOF >pod-env.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-env
namespace: dev
spec:
containers:
- name: busybox
image: busybox:1.30
command: ["/bin/sh","-c","while true;do /bin/echo $(date +%T);sleep 60; done;"]
env: # 设置环境变量列表
- name: "username"
value: "admin"
- name: "password"
value: "123456"
EOF
- 执行环境变量配置文件
[root@xnode1 ~]# kubectl create -f pod-env.yaml
pod/pod-env created
[root@xnode1 ~]# kubectl get pod -n dev
NAME READY STATUS RESTARTS AGE
pod-command 2/2 Running 0 13m
pod-env 1/1 Running 0 5s
pod-imagepullpolicy 1/2 CrashLoopBackOff 19 21h
#进入容器,输出环境变量
[root@xnode1 ~]# kubectl exec -it pod-env -n dev -c busybox /bin/sh
/ # echo $username
admin
/ # echo $password
123456
（6）、Pod端口配置
[root@xnode1 ~]# kubectl explain pod.spec.containers.ports | grep -Ev '^$'
KIND: Pod
VERSION: v1
RESOURCE: ports <[]Object>
DESCRIPTION:
List of ports to expose from the container. Exposing a port here gives the
system additional information about the network connections a container
uses, but is primarily informational. Not specifying a port here DOES NOT
prevent that port from being exposed. Any port which is listening on the
default "0.0.0.0" address inside a container will be accessible from the
network. Cannot be updated.
ContainerPort represents a network port in a single container.
FIELDS:
containerPort <integer> -required- #容器要监听的端口(0<x<65536)
Number of port to expose on the pod's IP address. This must be a valid port
number, 0 < x < 65536.
hostIP <string> #要将外部端口绑定到主机IP
What host IP to bind the external port to.
hostPort <integer> #容器要在主机上公开的端口,如果设置,主机上只能运行容器的一个副本
Number of port to expose on the host. If specified, this must be a valid
port number, 0 < x < 65536. If HostNetwork is specified, this must match
ContainerPort. Most containers do not need this.
name <string> #端口名称,如果指定,必须保证name在pod中是唯一的
If specified, this must be an IANA_SVC_NAME and unique within the pod. Each
named port in a pod must have a unique name. Name for the port that can be
referred to by services.
protocol <string> #端口协议:必须是TCP UDP协议或SCTP.默认是TCP
Protocol for port. Must be UDP, TCP, or SCTP. Defaults to "TCP".
- 编写一个pod-ports.yaml配置文件
[root@xnode1 ~]# cat <<EOF >pod-ports.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-ports
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports: # 设置容器暴露的端口列表
- name: nginx-port
containerPort: 80
protocol: TCP
EOF
- 查看是否允许
[root@xnode1 ~]# kubectl create -f pod-ports.yaml
pod/pod-ports created
[root@xnode1 ~]# kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
pod-command 2/2 Running 0 38m
pod-env 1/1 Running 0 25m
pod-imagepullpolicy 1/2 CrashLoopBackOff 24 22h
pod-ports 1/1 Running 0 29s
#查看pod-ports这个pod的yaml信息
[root@xnode1 ~]# kubectl get pods -n dev pod-ports -o yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: "2022-08-18T07:58:14Z"
name: pod-ports
namespace: dev
resourceVersion: "25418"
selfLink: /api/v1/namespaces/dev/pods/pod-ports
uid: 82f7f109-1ecb-11ed-a440-000c292ce9a5
spec:
containers:
- image: nginx:1.17.1
imagePullPolicy: IfNotPresent
name: nginx
ports:
- containerPort: 80 #映射的端口为80
name: nginx-port
protocol: TCP
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
#查看这个Pod的详细信息
[root@xnode1 ~]# kubectl describe pods -n dev pod-ports
Name: pod-ports
Namespace: dev
Priority: 0
PriorityClassName: <none>
Node: xnode2/10.0.0.40
Start Time: Thu, 18 Aug 2022 03:58:14 -0400
Labels: <none>
Annotations: <none>
Status: Running
IP: 10.244.1.47
Containers:
nginx:
Container ID: docker://b06bde3d33fb758a932e5980e660f5fb2355bb4a7a4649becfa15bab4b92054a
Image: nginx:1.17.1
Image ID: docker-pullable://nginx@sha256:b4b9b3eee194703fc2fa8afa5b7510c77ae70cfba567af1376a573a967c03dbb
Port: 80/TCP
Host Port: 0/TCP
State: Running
Started: Thu, 18 Aug 2022 03:58:15 -0400
Ready: True
Restart Count: 0
Environment: <none>
（7）、Pod的资源配额
什么是 资源配额?
容器中的程序运行对外提供服务,肯定是要占用一定的资源,比如CPU和内存,如果不对某个容器的资源做出限制,那么它就可能吃掉大量的资源,导致其它容器无法运行.针对这种情况,kubernetes提供了对内存和CPU的资源进行配额的机制,这种机制主要是通过resources选项实现的,他有两个子选项.
- limits: 用于限制运行时容器的最大占用资源,当容器占用资源超过 limits 时会被终止,并进行重启
- requests: 用于设置容器需要的最小资源,如果环境资源不够,容器将无法启动
- 编写一个测试案例yaml文件
[root@xnode1 ~]# cat <<EOF >pod-resources.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-resources
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
resources: # 资源配额
limits: # 限制资源（上限）
cpu: "2" # CPU限制，单位是core数
memory: "10Gi" # 内存限制
requests: # 请求资源（下限）
cpu: "1" # CPU限制，单位是core数
memory: "10Mi" # 内存限制
EOF
[root@xnode1 ~]# kubectl create -f pod-resources.yaml
pod/pod-resources created
- 查看是否在运行
[root@xnode1 ~]# kubectl get pod -n dev
NAME READY STATUS RESTARTS AGE
pod-command 2/2 Running 2 82m
pod-env 1/1 Running 1 70m
pod-ports 1/1 Running 1 45m
pod-resources 1/1 Running 0 7s
| CPU、MEM说明 |
|---|
| CPU: core数,可以做一个说明 |
| Memory： 内存大小，可以使用Gi、Mi、G、M等形式 |
（8）、Pod的生命周期
Pod的生命周期包括以下的过程:
- Pod创建过程
- 运行初始化容器(init container)过程
- 运行主容器(main container)过程
- 容器启动后钩子(post start)、容器终止前钩子（pre stop）
- 容器的存活性探测（Liveness probe）、就绪性探测（readiness probe）
- Pod终止过程
| 状态 | 动作 |
|---|---|
| 挂起（Pending）： | apiserver已经创建了pod资源对象，但它尚未被调度完成或者仍处于下载镜像的过程中 |
| 运行中（Running）： | pod已经被调度至某节点，并且所有容器都已经被kubelet创建完成 |
| 成功（Succeeded）： | pod中的所有容器都已经成功终止并且不会被重启 |
| 失败（Failed）： | 所有容器都已经终止，但至少有一个容器终止失败，即容器返回了非0值的退出状态 |
| 未知（Unknown）： | apiserver无法正常获取到pod对象的状态信息，通常由网络通信失败所导致 |
（9）、Pod的创建和终止
| Pod的创建和终止 |
|---|
| 1、用户通过kubectl或其它api客户端提交需要创建的pod信息给apiServer |
| 2、apiServer开始生成pod对象的信息，并将信息存入etcd，然后返回确认信息至客户端 |
| 3、apiServer开始反映etcd中的pod对象的变化，其它组件使用watch机制来跟踪检查apiServer上的变动 |
| 4、scheduler发现有新的pod对象要创建，开始为pod分配主机并将结果信息更新至apiServer |
| 5、node节点上的kubectl发现有pod调度过来，尝试调用docker启动容器，并将结果回送至apiServer |
| 6、apiServer将收到的pod状态信息存入etcd |
| Pod的终止过程 |
|---|
| 1、用户将APIServer发送删除pod对象的命令 |
| 2、apiServer中的pod对象信息会随着时间的推断而更新，在宽限期内（默认30s），Pod被视为dead |
| 3、将Pod标记为terminationing状态 |
| 4、kubelet在监控到pod对象转为terminating状态的同时启动pod关闭过程 |
| 5、端点控制器监控到Pod对象的关闭行为时将其从所有匹配到此端点的service资源的端口列表中移除 |
| 6、如果当前pod对象定义了preStop钩子处理器，则在其标记为terminating后即会同步到方式启动执行 |
| 7、Pod对象中的容器进程收到停止信号 |
| 8、宽限期结束后，若Pod中还存在仍在运行的进程，那么Pod对象会收到立即终止的信号 |
| 9、kubelet请求apiServer将此Pod资源的宽限期设置为0从而完成删除操作，此时Pod对于用户已不可见 |
（10）、初始化容器
初始化容器在Pod主容器之前要运行的容器,主要就是做一些主容器的前置工作,它具有两大特征:
- 初始化容器必须运行完成直至结束,若某初始化容器运行失败,那么kubernetes需要重启它直到成功完成
- 初始化容器必须按照定义的顺序执行,当且仅当前一个成功之后,后面的一个才能运行
初始化容器最长看到的应用场景:
- 提供主容器镜像中不具备的工具程序或自定义代码
- 初始化容器要先于2应用容器串行启动并运行完成,因此可用于延后应用容器的启动直至依赖的条件得到满足
- 例如:这里一个项目要求启动一个Nginx容器,但是启动Nginx的前置条件要先启动一个mysql容器,如果这个mysql容器没有启动那么这个Nginx容器就会一直处于初始化状态.
- 创建pod-initcontainer.yaml
[root@xnode1 ~]# cat <<EOF > pod-initcontainer.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-initcontainer
namespace: dev
spec:
containers:
- name: main-container
image: nginx:1.17.1
ports:
- name: nginx-port
containerPort: 80
initContainers:
- name: test-mysql
image: busybox:1.30
command: ['sh', '-c', 'until ping 192.168.5.14 -c 1 ; do echo waiting for mysql...; sleep 2; done;']
- name: test-redis
image: busybox:1.30
command: ['sh', '-c', 'until ping 192.168.5.15 -c 1 ; do echo waiting for reids...; sleep 2; done;']
EOF
- 启动并查看
[root@xnode1 ~]# kubectl create -f pod-initcontainer.yaml
pod/pod-initcontainer creat
[root@xnode1 ~]# kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
pod-initcontainer 0/1 Init:0/2 0 28sed
#可以看到容器一直卡在初始化没有运行
[root@xnode1 ~]# kubectl describe pod -n dev pod-initcontainer
... ...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 3m55s default-scheduler Successfully assigned dev/pod-initcontainer to xnode2
Normal Pulled 3m53s kubelet, xnode2 Container image "busybox:1.30" already present on machine
Normal Created 3m52s kubelet, xnode2 Created container test-mysql
Normal Started 3m52s kubelet, xnode2 Started container test-mysql
#动态监听pod
[root@xnode1 ~]# kubectl get pods -n dev -w
NAME READY STATUS RESTARTS AGE
pod-initcontainer 0/1 Init:0/2 0 112s
#新开一个终端为服务器新添加几个IP,观察pod的变化
[root@xnode1 ~]# ifconfig eno16777736:1 10.0.0.100 netmask 255.255.255.0 up
[root@xnode1 ~]# ifconfig eno16777736:1 10.0.0.200 netmask 255.255.255.0 up
#查看动态监听的pod:pod成功运行
[root@xnode1 ~]# kubectl get pod -n dev -w
NAME READY STATUS RESTARTS AGE
pod-initcontainer 0/1 Init:0/2 0 22s
pod-initcontainer 0/1 Init:1/2 0 2m27s
pod-initcontainer 0/1 Init:1/2 0 2m28s
pod-initcontainer 0/1 PodInitializing 0 2m40s
pod-initcontainer 1/1 Running 0 2m41s
（11）、钩子函数
钩子函数能够感知自身生命周期中的事件,并在相应的时刻到来之时运行用户程序指定的代码
kubernetes在主容器启动之后和之前提供了两个钩子函数:
- post start:容器创建之后执行,如果失败了会重启容器
- pre stop:容器终止之前执行,执行完成之后容器将成功终止,在其完成之前会阻塞删除容器的操作
- Exec命令:在容器内执行一次命令
……
lifecycle:
postStart:
exec:
command:
- cat
- /tmp/healthy
……
- TCPSocket：在当前容器尝试访问指定的socket
……
lifecycle:
postStart:
tcpSocket:
port: 8080
……
- HTTPGet：在当前容器中向某url发起http请求
……
lifecycle:
postStart:
httpGet:
path: / #URI地址
port: 80 #端口号
host: 192.168.5.3 #主机地址
scheme: HTTP #支持的协议，http或者https
……
以exec方式为例,演示钩子函数的使用,创建pod-hook-exec.yaml文件:
[root@xnode1 ~]# cat <<EOF > pod-hook-exec.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-hook-exec
namespace: dev
spec:
containers:
- name: main-container
image: nginx:1.17.1
ports:
- name: nginx-port
containerPort: 80
lifecycle:
postStart:
exec: # 在容器启动的时候执行一个命令，修改掉nginx的默认首页内容
command: ["/bin/sh", "-c", "echo postStart... > /usr/share/nginx/html/index.html"]
preStop:
exec: # 在容器停止之前停止nginx服务
command: ["/usr/sbin/nginx","-s","quit"]
EOF
- 创建Pod查看:
[root@xnode1 ~]# kubectl create -f pod-hook-exec.yaml
pod/pod-hook-exec created
[root@xnode1 ~]# kubectl get pods -n dev pod-hook-exec -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod-hook-exec 1/1 Running 0 49s 10.244.1.61 xnode2 <none> <none>
#访问这个IP:80端口
[root@xnode1 ~]# curl 10.244.1.61:80
postStart...
（12）、容器探测
容器探测 用于检测容器中的应用实例是否正常工作,是保障业务可用性的一种传统机制.经过探测,实例的状态不符合预期,那么kubernetes就会把该问题实例"摘除",不承担业务流量.kubernetes提供了两种探针来实现容器探测.
liveness Probes: 存活性探针,用于检测应用实例当前是否处于正常运行状态,如果不是,k8s会重启容器
readiness probes: 就绪性探针,用于检测应用实例当前是否可以接收请求,如果不能,k8s不会转发流量
这两者之间不同的区别是,决定是否重启容器,readinessProbe决定是否将请求转发给容器.
上面的这两者探针均支持3种探测方式.
- exec命令: 在容器内执行一次命令,如果命令执行的退出码为0,则认为程序正常,否则不正常.
……
livenessProbe:
exec:
command:
- cat
- /tmp/healthy
……
- TCPSocket: 将会尝试访问一个用户容器的端口，如果能够建立这条连接，则认为程序正常，否则不正常.
……
livenessProbe:
tcpSocket:
port: 8080
……
- HTTPGet: 调用容器内Web应用的URL，如果返回的状态码在200和399之间，则认为程序正常，否则不正常
……
livenessProbe:
httpGet:
path: / #URI地址
port: 80 #端口号
host: 127.0.0.1 #主机地址
scheme: HTTP #支持的协议，http或者https
……
以 liveness probes 为例,做几个演示:
方式一: Exec
- 创建一个pod-liveness-exec.yaml
[root@xnode1 ~]# cat <<EOF > pod-liveness-exec.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-liveness-exec
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- name: nginx-port
containerPort: 80
livenessProbe:
exec:
command: ["/bin/cat","/tmp/hello.txt"] # 执行一个查看文件的命令
EOF
#创建pod并查看状态:
[root@xnode1 ~]# kubectl create -f pod-liveness-exec.yaml
pod/pod-liveness-exec created
[root@xnode1 ~]# kubectl get pods -n dev pod-liveness-exec
NAME READY STATUS RESTARTS AGE
pod-liveness-exec 1/1 Running 0 23s
#查看状态:发现已经重启4次
[root@xnode1 ~]# kubectl get pods -n dev pod-liveness-exec
NAME READY STATUS RESTARTS AGE
pod-liveness-exec 1/1 Running 4 2m5s
#查看这个pod的详细信息:发现这个Pod在执行livenessProbe探测,内容是使用command查看/tmp目录下的Hello文件,没有查找到这个文件,没有达到这个预期于是一直重启容器
[root@xnode1 ~]# kubectl describe pods -n dev pod-liveness-exec
... ...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 70s default-scheduler Successfully assigned dev/pod-liveness-exec to xnode2
Normal Pulled 9s (x3 over 69s) kubelet, xnode2 Container image "nginx:1.17.1" already present on machine
Normal Created 9s (x3 over 68s) kubelet, xnode2 Created container nginx
Warning Unhealthy 9s (x6 over 59s) kubelet, xnode2 Liveness probe failed: /bin/cat: /tmp/hello.txt: No such file or directory
Normal Killing 9s (x2 over 39s) kubelet, xnode2 Container nginx failed liveness probe, will be restarted
Normal Started 8s (x3 over 67s) kubelet, xnode2 Started container nginx
- 我们这里修改pod-liveness的yaml文件: command修改为查看/tmp目录下的配置文件
[root@xnode1 ~]# cat <<EOF > pod-liveness-exec.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-liveness-exec
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- name: nginx-port
containerPort: 80
livenessProbe:
exec:
command: ["/bin/ls","/tmp/"] # 执行一个查看文件的命令
EOF
#执行创建这个Pod并查看状态:发现容器没有重启,查看容器的详细信息都是一次性通过
[root@xnode1 ~]# kubectl create -f pod-liveness-exec.yaml
pod/pod-liveness-exec created
[root@xnode1 ~]# kubectl get pods -n dev pod-liveness-exec
NAME READY STATUS RESTARTS AGE
pod-liveness-exec 1/1 Running 0 14s
[root@xnode1 ~]# kubectl describe pod -n dev pod-liveness-exec
... ...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 66s default-scheduler Successfully assigned dev/pod-liveness-exec to xnode2
Normal Pulled 65s kubelet, xnode2 Container image "nginx:1.17.1" already present on machine
Normal Created 64s kubelet, xnode2 Created container nginx
Normal Started 64s kubelet, xnode2 Started container nginx
方式二: TCP Socket
#创建一个pod-liveness-tcpsocket.yaml
[root@xnode1 ~]# cat <<EOF > pod-liveness-tcpsocket.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-liveness-tcpsocket
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- name: nginx-port
containerPort: 80
livenessProbe:
tcpSocket:
port: 8080 # 尝试访问8080端口
EOF
#创建pod 查看pod状态:由于pod容器创建初执行的liveness容器探测要求访问8080端口,pod无法访问这个端口于是该容器执行重启
[root@xnode1 ~]# kubectl create -f pod-liveness-tcpsocket.yaml
pod/pod-liveness-tcpsocket created
[root@xnode1 ~]# kubectl get pods -n dev pod-liveness-tcpsocket
NAME READY STATUS RESTARTS AGE
pod-liveness-tcpsocket 1/1 Running 1 35s
[root@xnode1 ~]# kubectl describe pod -n dev pod-liveness-tcpsocket
... ...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 56s default-scheduler Successfully assigned dev/pod-liveness-tcpsocket to xnode2
Normal Pulled 28s (x2 over 55s) kubelet, xnode2 Container image "nginx:1.17.1" already present on machine
Normal Killing 28s kubelet, xnode2 Container nginx failed liveness probe, will be restarted
Normal Created 27s (x2 over 55s) kubelet, xnode2 Created container nginx
Normal Started 27s (x2 over 55s) kubelet, xnode2 Started container nginx
Warning Unhealthy 8s (x5 over 48s) kubelet, xnode2 Liveness probe failed: dial tcp 10.244.1.64:8080: connect: connection refused
- 我们这里修改一下pod-liveness-tcpsocket这个容器的配置文件 :
[root@xnode1 ~]# cat <<EOF > pod-liveness-tcpsocket.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-liveness-tcpsocket
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- name: nginx-port
containerPort: 80
livenessProbe:
tcpSocket:
port: 80 # 尝试访问80端口
EOF
#查看:发现既没有重启也没有报错
[root@xnode1 ~]# kubectl create -f pod-liveness-tcpsocket.yaml
pod/pod-liveness-tcpsocket created
[root@xnode1 ~]#
[root@xnode1 ~]# kubectl get pod -n dev pod-liveness-tcpsocket
NAME READY STATUS RESTARTS AGE
pod-liveness-tcpsocket 1/1 Running 0 9s
[root@xnode1 ~]# kubectl describe pods -n dev pod-liveness-tcpsocket
... ...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 38s default-scheduler Successfully assigned dev/pod-liveness-tcpsocket to xnode2
Normal Pulled 37s kubelet, xnode2 Container image "nginx:1.17.1" already present on machine
Normal Created 37s kubelet, xnode2 Created container nginx
Normal Started 37s kubelet, xnode2 Started container nginx
方式三:httpGET
[root@xnode1 ~]# cat <<EOF > pod-liveness-httpget.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-liveness-httpget
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- name: nginx-port
containerPort: 80
livenessProbe:
httpGet: # 其实就是访问http://127.0.0.1:80/hello
scheme: HTTP #支持的协议，http或者https
port: 80 #端口号
path: /hello #URI地址
EOF
#查看
[root@xnode1 ~]# kubectl create -f pod-liveness-httpget.yaml
pod/pod-liveness-httpget created
[root@xnode1 ~]# kubectl get pods -n dev pod-liveness-httpget
NAME READY STATUS RESTARTS AGE
pod-liveness-httpget 1/1 Running 3 113s
#查看pod的详细信息:访问地址404 not found
[root@xnode1 ~]# kubectl describe pod -n dev pod-liveness-httpget
... ...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 103s default-scheduler Successfully assigned dev/pod-liveness-httpget to xnode2
Normal Pulled 20s (x4 over 102s) kubelet, xnode2 Container image "nginx:1.17.1" already present on machine
Normal Killing 20s (x3 over 80s) kubelet, xnode2 Container nginx failed liveness probe, will be restarted
Normal Created 19s (x4 over 102s) kubelet, xnode2 Created container nginx
Normal Started 19s (x4 over 101s) kubelet, xnode2 Started container nginx
Warning Unhealthy 10s (x10 over 100s) kubelet, xnode2 Liveness probe failed: HTTP probe failed with statuscode: 404
- 我们这里修改一下pod-liveness-httpget这个容器的配置文件 :
[root@xnode1 ~]# cat <<EOF > pod-liveness-httpget.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-liveness-httpget
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- name: nginx-port
containerPort: 80
livenessProbe:
httpGet: # 其实就是访问http://127.0.0.1:80/hello
scheme: HTTP #支持的协议，http或者https
port: 80 #端口号
path: / #URI地址
EOF
#再次验证查看:发现没有进行重启,describe查看详细信息也没有报错
[root@xnode1 ~]# kubectl get pods -n dev pod-liveness-httpget
NAME READY STATUS RESTARTS AGE
pod-liveness-httpget 1/1 Running 0 57s
[root@xnode1 ~]# kubectl get pods -n dev pod-liveness-httpget
... ...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 27s default-scheduler Successfully assigned dev/pod-liveness-httpget to xnode2
Normal Pulled 26s kubelet, xnode2 Container image "nginx:1.17.1" already present on machine
Normal Created 26s kubelet, xnode2 Created container nginx
Normal Started 26s kubelet, xnode2 Started container nginx
至此已经使用了liveness Probe演示了3种探测方式,但是查看liveness Probe的子属性,会发现除了这三种方式,还有一些其它的配置,这里进行一一解释.
[root@xnode1 ~]# kubectl explain pod.spec.containers.livenessProbe
FIELDS:
exec <Object>
tcpSocket <Object>
httpGet <Object>
initialDelaySeconds <integer> # 容器启动后等待多少秒执行第一次探测
timeoutSeconds <integer> # 探测超时时间。默认1秒，最小1秒
periodSeconds <integer> # 执行探测的频率。默认是10秒，最小1秒
failureThreshold <integer> # 连续探测失败多少次才被认定为失败。默认是3。最小值是1
successThreshold <integer> # 连续探测成功多少次才被认定为成功。默认是1
（13）、重启策略
一旦容器出现了问题,kubernetes就会对容器所在的Pod进行重启,其实这是有Po的重启策略决定的,pod的重启策略有3种,分别如下:
Always: 容器失效时,自动重启该容器,这也就是默认值
OnFailure: 容器终止运行且退出码不为0时重启
Never: 无论状态如何,都不重启该容器
重启策略适用于pod对象中的所有容器，首次需要重启的容器，将在其需要时立即进行重启，随后再次需要重启的操作将由kubelet延迟一段时间后进行，且反复的重启操作的延迟时长以此为10s、20s、40s、80s、160s和300s，300s是最大延迟时长。
- 创建 Pod-restartpolicy.yaml
[root@xnode1 ~]# cat <<EOF > pod-restartpolicy.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-restartpolicy
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- name: nginx-port
containerPort: 80
livenessProbe:
httpGet:
scheme: HTTP
port: 80
path: /hello
restartPolicy: Always # 设置重启策略为Always
EOF
#查看状态:
[root@xnode1 ~]# kubectl create -f pod-restartpolicy.yaml
pod/pod-restartpolicy created
[root@xnode1 ~]# kubectl get pods -n dev pod-restartpolicy -w
NAME READY STATUS RESTARTS AGE
pod-restartpolicy 1/1 Running 0 25s
pod-restartpolicy 1/1 Running 1 31s
#我们修改配置文件将重启策略修改成"Never",不管遇到什么状态永不重启
[root@xnode1 ~]# cat <<EOF > pod-restartpolicy.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-restartpolicy
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- name: nginx-port
containerPort: 80
livenessProbe:
httpGet:
scheme: HTTP
port: 80
path: /hello
restartPolicy: Never # 设置重启策略为Never
EOF
#再次启动并查看状态
[root@xnode1 ~]# kubectl create -f pod-restartpolicy.yaml
pod/pod-restartpolicy created
[root@xnode1 ~]# kubectl get pods -n dev pod-restartpolicy -w
NAME READY STATUS RESTARTS AGE
pod-restartpolicy 1/1 Running 0 10s
pod-restartpolicy 0/1 Completed 0 28s
[root@xnode1 ~]# kubectl describe pods -n dev pod-restartpolicy
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 2m default-scheduler Successfully assigned dev/pod-restartpolicy to xnode2
Normal Pulled 119s kubelet, xnode2 Container image "nginx:1.17.1" already present on machine
Normal Created 119s kubelet, xnode2 Created container nginx
Normal Started 119s kubelet, xnode2 Started container nginx
Warning Unhealthy 93s (x3 over 113s) kubelet, xnode2 Liveness probe failed: HTTP probe failed with statuscode: 404
Normal Killing 93s kubelet, xnode2 Stopping container nginx
十、Pod的调度
默认情况下,一个Pod在哪个节点上运行,是由Scheduler组件采用相应的算法算出来的,这个时候是不受人为干预,但是在实际使用中这并不能满足我们的需求,在很多情况下我们向控制某个Pod达到某些节点上,那么应该怎么做呢？这就要求了解kubernetes对Pod的调度规则，kubernetes提供了四大类调度方式：
- 自动调度：运行在哪个节点上完全由Scheduler经过一系列的算法计算得出
- 定向调度：NodeName、NodeSelector
- 亲和性调度：NodeAffinity、PodAffinity、PodAntiAffinity
- 污点（容忍）调度：Taints、Toleration
（1）、定向调度:
定向调度:
定向调度, 指的是利用在Pod上声明nodename或者nodeSelector,从此将Pod调度到期望值的node节点上,注意,这里的调度是强制的,这就意味着即使调度的目标node不在,也会向上面进行调度,只不过pod运行失败而已.
NodeName:
用于强制约束将Pod调度到指定的name的node节点上.这种方式,其实是直接跳过Scheduler的调度逻辑 ,直接将pod调度到指定名称的节点.
- 实验一下创建一个pod-nodename.yaml文件
[root@xnode1 ~]# cat <<EOF > pod-nodename.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-nodename
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
nodeName: xnode2 # 指定调度到node2节点上
EOF
#创建Pod并查看Pod详细信息
[root@xnode1 ~]# kubectl create -f pod-nodename.yaml
pod/pod-nodename created
[root@xnode1 ~]# kubectl get pods -n dev pod-nodename -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod-nodename 1/1 Running 0 20s 10.244.1.79 xnode2 <none> <none>
（2）、NodeSelector
NodeSelector 用于将pod调度到添加了指定标签的node节点上,它是通过kubernetes的label-selector实现的,也就是说,在Pod创建之前,会由scheduler使用MatchSelector调度策略来进行label匹配,找出目标Node,然后将pod调度到目标节点,该匹配着是强制约束.
- 我们这里用实验来验证一下:
#首先分别给xnode节点添加标签:
[root@xnode1 ~]# kubectl label nodes xnode1 nodeenv=pro
node/xnode1 labeled
[root@xnode1 ~]# kubectl label nodes xnode2 nodeenv=test
node/xnode2 labeled
#创建一个pod-nodeselector.yaml文件,文件中声明一个键值对进行目标打了标签的node进行筛选,node使用它创建一个Pod
[root@xnode1 ~]# cat <<EOF > pod-nodeselector.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-nodeselector
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
nodeSelector:
nodeenv: pro # 指定调度到具有nodeenv=pro标签的节点上
EOF
#我们这里查看一下pod的状态:因为xnode1是master节点不会运行用户级别的普通pod,所以调度到xnode1上,xnode1拒绝了
[root@xnode1 ~]# kubectl get pod -n dev pod-nodeselector -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod-nodeselector 0/1 Pending 0 101s <none> <none> <none> <none>
[root@xnode1 ~]# kubectl describe pod -n dev pod-nodeselector
... ...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 50s (x2 over 50s) default-scheduler 0/2 nodes are available: 1 node(s) didn't match node selector, 1 node(s) had taints that the pod didn't tolerate.
（3）、亲和性调度
上面讲了定向调度,使用起来很方便,但是也有问题,那就是如果没有满足条件的Node,那么Pod将不会运行,即使在集群中还有可用Node列表也不行,这样就限制了它的使用场景.
基于上面的这些问题kubernetes提出来一种亲和性调度(Affinity)作为解决方案,它在NodeSelector的基础上进行了扩展,可以通过配置的形式,实现优先选择满足node进行调度,如果没有,也可以调度到不满足条件的节点上,使扩展更加灵活.
Affinity分为三大类:
- node Affinity(node亲和性) : 以node为目标,解决Pod可以调度到哪些node的问题
- pod Affinity(pod亲和性) : 以pod为目标,解决pod可以和哪些已存在的pod部署在同一个拓扑域中的问题.
- pod AntiAffinity(pod反亲和性): 以Pod为目标,决绝pod不能和哪些已存在pod部署在同一拓扑域中的问题.
使用场景说明:
- 亲和性: 如果两个应用需要频繁的进行网络通信交互,这样就很有必要利用亲和性让两个应用尽可能的靠近,这样可以减少因网络通信而带来的性能损耗.
- 反亲和性: 当应用采用多副本部署时,有必要采用反亲和性让各个应用实例打散分布在各个node上,这样可以提高服务的高可用性和扩展性
- NodeAffinity: 首先来看一下
NodeAffinity的可配置项：
pod.spec.affinity.nodeAffinity
requiredDuringSchedulingIgnoredDuringExecution Node节点必须满足指定的所有规则才可以，相当于硬限制
nodeSelectorTerms 节点选择列表
matchFields 按节点字段列出的节点选择器要求列表
matchExpressions 按节点标签列出的节点选择器要求列表(推荐)
key 键
values 值
operator 关系符 支持Exists, DoesNotExist, In, NotIn, Gt, Lt
preferredDuringSchedulingIgnoredDuringExecution 优先调度到满足指定的规则的Node，相当于软限制 (倾向)
preference 一个节点选择器项，与相应的权重相关联
matchFields 按节点字段列出的节点选择器要求列表
matchExpressions 按节点标签列出的节点选择器要求列表(推荐)
key 键
values 值
operator 关系符 支持In, NotIn, Exists, DoesNotExist, Gt, Lt
weight 倾向权重，在范围1-100。
关系符的使用说明:
- matchExpressions:
- key: nodeenv # 匹配存在标签的key为nodeenv的节点
operator: Exists
- key: nodeenv # 匹配标签的key为nodeenv,且value是"xxx"或"yyy"的节点
operator: In
values: ["xxx","yyy"]
- key: nodeenv # 匹配标签的key为nodeenv,且value大于"xxx"的节点
operator: Gt
values: "xxx"
- 接下来首先演示一下
requiredDuringSchedulingIgnoredDuringExecution硬限制,
#创建一个pod-nodeaffinity-required.yaml
[root@xnode1 ~]# cat <<EOF > pod-nodeaffinity-required.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-nodeaffinity-required
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
affinity: #亲和性设置
nodeAffinity: #设置node亲和性
requiredDuringSchedulingIgnoredDuringExecution: # 硬限制
nodeSelectorTerms:
- matchExpressions: # 匹配env的值在["xxx","yyy"]中的标签
- key: nodeenv
operator: In
values: ["xxx","yyy"]
EOF
#查看状态
[root@xnode1 ~]# kubectl create -f pod-nodeaffinity-required.yaml
pod/pod-nodeaffinity-required created
[root@xnode1 ~]#
[root@xnode1 ~]# kubectl get pods -n dev pod-nodeaffinity-required -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod-nodeaffinity-required 0/1 Pending 0 17s <none> <none> <none> <none>
#查看这个pod的详细信息:Waring提示目前两个节点都不能满足您的调度需求
[root@xnode1 ~]# kubectl describe pod -n dev pod-nodeaffinity-required
... ...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 49s default-scheduler 0/2 nodes are available: 2 node(s) didn't match node selector.
#重新修改这个配置文件
[root@xnode1 ~]# cat <<EOF > pod-nodeaffinity-required.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-nodeaffinity-required
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
affinity: #亲和性设置
nodeAffinity: #设置node亲和性
requiredDuringSchedulingIgnoredDuringExecution: # 硬限制
nodeSelectorTerms:
- matchExpressions: # 匹配env的值在["xxx","yyy"]中的标签
- key: nodeenv
operator: In
values: ["test","yyy"]
EOF
#再次尝试
[root@xnode1 ~]# kubectl create -f pod-nodeaffinity-required.yaml
pod/pod-nodeaffinity-required created
[root@xnode1 ~]# kubectl get pods -n dev -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod-nodeaffinity-required 1/1 Running 0 6s 10.244.1.80 xnode2 <none> <none>
pod-nodename 1/1 Running 0 86m 10.244.1.79 xnode2 <none> <none>
- 我们来演示一下
requiredDuringSchedulingIgnoredDuringExecution,软限制
[root@xnode1 ~]# cat <<EOF > pod-nodeaffinity-preferred.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-nodeaffinity-preferred
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
affinity: #亲和性设置
nodeAffinity: #设置node亲和性
preferredDuringSchedulingIgnoredDuringExecution: # 软限制
- weight: 1
preference:
matchExpressions: # 匹配env的值在["xxx","yyy"]中的标签(当前环境没有)
- key: nodeenv
operator: In
values: ["xxx","yyy"]
EOF
[root@xnode1 ~]# kubectl get pods -n dev pod-nodeaffinity-preferred -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod-nodeaffinity-preferred 1/1 Running 0 31s 10.244.1.81 xnode2 <none> <none>
#因为使用的调度方法为软限制,软限制的调度策略优先寻找匹配的values,如果找不到就调度到可用的node节点上
[root@xnode1 ~]# kubectl describe pod -n dev pod-nodeaffinity-preferred
... ...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 56s default-scheduler Successfully assigned dev/pod-nodeaffinity-preferred to xnode2
Normal Pulled 55s kubelet, xnode2 Container image "nginx:1.17.1" already present on machine
Normal Created 54s kubelet, xnode2 Created container nginx
Normal Started 54s kubelet, xnode2 Started container nginx
NodeAffinity注意事项:
NodeAffinity规则设置的注意事项:
①.如果同时定义了nodeSelector(指定node标签匹配)和nodeAffinity(指定node匹配),那么必须两个条件都得到满足,Pod才能运行在指定的Node上.
②.如果nodeAffinity(指定node匹配)指定了多个nodeSelectorTerms(指定node标签匹配),那么只需要其中一个能够匹配成功即可
③.如果一个nodeSelectorTerms中有多个matchExpressions,则一个节点必须满足所有的才能匹配成功
④.如果一个Pod所在的Node在Pod运行期间其标签发生了变化,不再符合该Pod的节点亲和性需求,则系统将忽略此变化
（4）、PodAffinity
PodAffinity 主要实现以运行的Pod为参照物,实现让新创建的Pod跟参照Pod在一个区域的功能.
我们查看一下
PodAffinity的可配置选项:
pod.spec.affinity.podAffinity
# requiredDuringSchedulingIgnoredDuringExecution 硬限制
namespaces 指定参照pod的namespace
topologyKey 指定调度作用域
labelSelector 标签选择器
matchExpressions 按节点标签列出的节点选择器要求列表(推荐)
key 键
values 值
operator 关系符 支持In, NotIn, Exists, DoesNotExist.
matchLabels 指多个matchExpressions映射的内容
# preferredDuringSchedulingIgnoredDuringExecution 软限制
podAffinityTerm 选项
namespaces
topologyKey
labelSelector
matchExpressions
key 键
values 值
operator
matchLabels
weight 倾向权重，在范围1-100
toplogKey用于指定调度时作用域,例如:
- 如果指定kubernetes.io/hostname,那就是以Node节点为分区范围
- 如果指定为beta.kubernetes.io/os,则以Node节点的操作系统类型来区分
- 演示
requiredDuringSchedulingIgnoredDuringExecution - (1)、首先 创建一个参照Pod，pod-podaffinity-target.yaml
[root@xnode1 ~]# cat <<EOF > pod-podaffinity-target.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-podaffinity-target
namespace: dev
labels:
podenv: test #设置标签
spec:
containers:
- name: nginx
image: nginx:1.17.1
nodeName: xnode2 # 将目标pod名确指定到xnode2上
EOF
#查看Pod的信息
[root@xnode1 ~]# kubectl get pods -n dev pod-podaffinity-target -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod-podaffinity-target 1/1 Running 0 40s 10.244.1.89 xnode2 <none>
[root@xnode1 ~]# kubectl get pods -n dev pod-podaffinity-target -o wide --show-labels
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES LABELS
pod-podaffinity-target 1/1 Running 0 71s 10.244.1.89 xnode2 <none> <none> podenv=test<none>
- (2)、创建
pod-podaffinity-required.yaml，内容如下：
[root@xnode1 ~]# cat <<EOF > pod-podaffinity-required.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-podaffinity-required
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
affinity: #亲和性设置
podAffinity: #设置pod亲和性
requiredDuringSchedulingIgnoredDuringExecution: # 硬限制
- labelSelector:
matchExpressions: # 匹配env的值在["xxx","yyy"]中的标签
- key: podenv
operator: In
values: ["xxx","yyy"]
topologyKey: kubernetes.io/hostname
EOF
#启动pod
[root@xnode1 ~]# kubectl create -f pod-podaffinity-required.yaml
pod/pod-podaffinity-required created
#查看容器状态发现启动失败:查看详情"两个节点不可用,一个节点与pod关联规则不匹配,一个节点不匹配pod关联,一个节点有污点"
[root@xnode1 ~]# kubectl get pods -n dev pod-podaffinity-required -o wide --show-labels
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES LABELS
pod-podaffinity-required 0/1 Pending 0 60s <none> <none> <none> <none> <none>
[root@xnode1 ~]# kubectl describe pod -n dev pod-podaffinity-required
... ...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 16s (x3 over 105s) default-scheduler 0/2 nodes are available: 1 node(s) didn't match pod affinity rules, 1 node(s) didn't match pod affinity/anti-affinity, 1 node(s) had taints that the pod didn't tolerate.
#修改配置文件
[root@xnode1 ~]# cat <<EOF > pod-podaffinity-required.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-podaffinity-required
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
affinity: #亲和性设置
podAffinity: #设置pod亲和性
requiredDuringSchedulingIgnoredDuringExecution: # 硬限制
- labelSelector:
matchExpressions: # 匹配env的值在["test","yyy"]中的标签
- key: podenv
operator: In
values: ["test","yyy"]
topologyKey: kubernetes.io/hostname
EOF
#再次查看
[root@xnode1 ~]# kubectl get pods -n dev pod-podaffinity-required -o wide --show-labels
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES LABELS
pod-podaffinity-required 1/1 Running 0 6s 10.244.1.90 xnode2 <none> <none> <none>
（5）、PodAntiAffinity
PodAntiAffinity 主要实现以运行的Pod为参照,让新创建的Pod跟参照Pod不再一个区域中的功能.
- （1）、继续使用上面的案例
[root@k8s-master01 ~]# kubectl get pods -n dev -o wide --show-labels
NAME READY STATUS RESTARTS AGE IP NODE LABELS
pod-podaffinity-required 1/1 Running 0 3m29s 10.244.1.38 node1 <none>
pod-podaffinity-target 1/1 Running 0 9m25s 10.244.1.37 node1 podenv=pro
- （2）、创建pod-podantiaffinity-required.yaml，内容如下：
# 上面配置表达的意思是：新Pod必须要与拥有标签nodeenv=pro的pod不在同一Node上，运行测试一下。
apiVersion: v1
kind: Pod
metadata:
name: pod-podantiaffinity-required
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
affinity: #亲和性设置
podAntiAffinity: #设置pod亲和性
requiredDuringSchedulingIgnoredDuringExecution: # 硬限制
- labelSelector:
matchExpressions: # 匹配podenv的值在["pro"]中的标签
- key: podenv
operator: In
values: ["pro"]
topologyKey: kubernetes.io/hostname
# 创建pod
[root@k8s-master01 ~]# kubectl create -f pod-podantiaffinity-required.yaml
pod/pod-podantiaffinity-required created
# 查看pod
# 发现调度到了node2上
[root@k8s-master01 ~]# kubectl get pods pod-podantiaffinity-required -n dev -o wide
NAME READY STATUS RESTARTS AGE IP NODE ..
pod-podantiaffinity-required 1/1 Running 0 30s 10.244.1.96 node2 ..
（6）、调度污点
污点(Taints):
在前面的调度方式都是站在Pod的角度上,通过在Pod上添加属性,来确定Pod是否要调度到指定的Node上,其实我们也可以站在Node的角度上,通过在Node上添加 污点 属性,来决定是否允许Pod调度过来.
Node被设置上污点之后就和Pod之间存在了一种相斥的关系,进而拒绝Pod调度进来,甚至可以将已存在的Pod驱逐出去;污点的格式为:
key=value:effect,key和value是污点的标签,effect描述污点的作用,支持如下三个选项:
- **PreferNoSchedule: ** kubernetes将尽量避免把Pod调度到具有该污点的Pod上,除非没有其它节点可以调度
- NoSchedule: kubernetes将不会把Pod调度到具有该污点的Node上,但不会影响当前Node上已存在的Pod
- NoExecute: kubernetes将不会把Pod调度到具有该污点的Node上,同时也会将Node上已存在的Pod驱离.
- 设置污点、去处污点
# 设置污点
kubectl taint nodes xnode1 key=value:effect
# 去除污点
kubectl taint nodes xnode1 key:effect-
# 去除所有污点
kubectl taint nodes xnode1 key-
- 接下来,演示下污点的效果:
- 准备节点node1（为了演示效果更加明显，暂时停止node2节点）
- 为node1节点设置一个污点:
tag=heima:PreferNoSchedule；然后创建pod1( pod1 可以 ) - 修改为node1节点设置一个污点:
tag=heima:NoSchedule；然后创建pod2( pod1 正常 pod2 失败 ) - 修改为node1节点设置一个污点:
tag=heima:NoExecute；然后创建pod3 ( 3个pod都失败 )
#为xnode2和xnode3设置污点
[root@xnode1 ~]# kubectl taint nodes xnode2 tag=xnode2:NoSchedule
node/xnode2 tainted
[root@xnode1 ~]# kubectl describe node xnode2 | grep Taints:
Taints: tag=xnode2:NoSchedule
[root@xnode1 ~]# kubectl taint nodes xnode3 tag=xnode3:PreferNoSchedule
node/xnode3 tainted
[root@xnode1 ~]# kubectl describe node xnode3 | grep Taints:
Taints: tag=xnode3:PreferNoSchedule
#启动Pod
[root@xnode1 ~]# kubectl run taint1 --image=nginx:1.17.1 -n dev
kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
deployment.apps/taint1 created
#查看Pod:xnode2设置为NoSchedule(新的不要来,旧的不动),xnode3设置为PreferNoSchedule(尽量别来)
#此时的Pod应运行在xnode3上
[root@xnode1 ~]# kubectl get pod -n dev -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
taint1-56b4468b94-4bd2m 1/1 Running 0 2m11s 10.244.2.6 xnode3 <none> <none>
#把xnode3设置成NoSchedule(新的不要来,旧的不动)
#把xnode2设置成PreferNoSchedule(尽量别来)
[root@xnode1 ~]# kubectl taint nodes xnode3 tag=xnode3:NoSchedule
node/xnode3 tainted
[root@xnode1 ~]# kubectl taint nodes xnode2 tag=xnode2:PreferNoSchedule
node/xnode2 tainted
#再次启动Pod
#可以看到xnode3的容器正在运行,新Pod不会调度到xnode3
[root@xnode1 ~]# kubectl get pod -n dev -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
taint1-56b4468b94-4bd2m 1/1 Running 0 10m 10.244.2.6 xnode3 <none> <none>
taint2-88556f6-8r47g 1/1 Running 0 3m24s 10.244.1.11 xnode2 <none> <none>
#将xnode3设置成NoExecute(新的不要来啊旧的赶紧走)
[root@xnode1 ~]# kubectl taint nodes xnode3 tag:NoSchedule-
node/xnode3 untainted
[root@xnode1 ~]# kubectl taint nodes xnode3 tag=xnode3:NoExecute
node/xnode3 tainted
#再次查看Pod:发现Pod taint1转移到xnode2
[root@xnode1 ~]# kubectl get pod -n dev -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
taint1-56b4468b94-62xmb 1/1 Running 0 13s 10.244.1.12 xnode2 <none> <none>
taint2-88556f6-8r47g 1/1 Running 0 6m19s 10.244.1.11 xnode2 <none> <none>
（7）、污点的容忍（Toleration）
容忍(Toleration):
在node上添加污点用于拒绝pod调度上来,但是如果就是想将一个pod调度到一个有污点的node上去,这时候应该怎么做呢?这个实施就要使用到容忍.
污点就是拒绝,容忍就是忽略,Node通过污点拒绝pod调度上去,Pod通过容忍忽略拒绝.
- 在之前xnode3已经打上了NoExecute的污点,此时Pod是调度不上去的,关闭xnode2.
- 这里可以通过给pod添加容忍,然后将其调度上去
创建 pod-toleration.yaml文件
cat <<EOF > pod-toleration.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-toleration
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
tolerations: # 添加容忍
- key: "tag" # 要容忍的污点的key
operator: "Equal" # 操作符
value: "xnode3" # 容忍的污点的value
effect: "NoExecute" # 添加容忍的规则，这里必须和标记的污点规则相同
EOF
#没有设置容忍直接启动Pod:新启动的Pod状态为Pending
[root@xnode1 ~]# kubectl run taint3 --image=nginx:1.17.1 -n dev
kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
deployment.apps/taint3 created
[root@xnode1 ~]# kubectl get pods -n dev -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
taint1-56b4468b94-62xmb 1/1 Running 0 29m 10.244.1.12 xnode2 <none> <none>
taint2-88556f6-8r47g 1/1 Running 0 35m 10.244.1.11 xnode2 <none> <none>
taint3-69b4df4bf4-gzf9l 0/1 Pending 0 12s <none> <none> <none> <none>
#使用设置了污点容忍的yaml文件创建一个Pod:运行状态正常
[root@xnode1 ~]# kubectl create -f pod-toleration.yaml
pod/pod-toleration created
[root@xnode1 ~]# kubectl get pods -n dev pod-toleration -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod-toleration 1/1 Running 0 53s 10.244.2.7 xnode3 <none> <none>
- 容忍的详细配置
kubectl explain pod.spec.tolerations
......
FIELDS:
key # 对应着要容忍的污点的键，空意味着匹配所有的键
value # 对应着要容忍的污点的值
operator # key-value的运算符，支持Equal和Exists（默认）
effect # 对应污点的effect，空意味着匹配所有影响
tolerationSeconds # 容忍时间, 当effect为NoExecute时生效，表示pod在Node上的停留时间
十一、Pod控制器
Pod控制器介绍:
在kubernetes中,按照Pod的创建方式可以分为两种
- **自主式Pod:**kubernetes直接创建出来的Pod,这种Pod删除后就没有了,也不会重建
- 控制器创建的Pod: 通过控制器创建的Pod,这种Pod删除了之后还会自动重建
Pod控制器是管理pod的中间层，使用Pod控制器之后，只需要告诉Pod控制器，想要多少个什么样的Pod就可以了，它会创建出满足条件的Pod并确保每一个Pod资源处于用户期望的目标状态。如果Pod资源在运行中出现故障，它会基于指定策略重新编排Pod。
- 在kubernetes中,有很多类型的Pod控制器,每种都有自己的适合的场景,常见的有下面这些:
| Pod控制器的类型 | 作用 |
|---|---|
| ReplicationController： | 比较原始的pod控制器，已经被废弃，由ReplicaSet替代 |
| ReplicaSet： | 保证副本数量一直维持在期望值，并支持pod数量扩缩容，镜像版本升级 |
| Deployment： | 通过控制ReplicaSet来控制Pod，并支持滚动升级、回退版本 |
| Horizontal Pod Autoscaler： | 可以根据集群负载自动水平调整Pod的数量，实现削峰填谷 |
| DaemonSet： | 在集群中的指定Node上运行且仅运行一个副本，一般用于守护进程类的任务 |
| Job： | 它创建出来的pod只要完成任务就立即退出，不需要重启或重建，用于执行一次性任务 |
| Cronjob： | 它创建的Pod负责周期性任务控制，不需要持续后台运行 |
| StatefulSet： | 管理有状态应用 |
（1）、ReplicaSet（RS）
ReplicaSet的主要作用是 保证一定数量的Pod正常运行 ，它会持续监听这些Pod的运行状态，一旦Pod发生故障，就会重启或重建。同时它还支持对Pod数量的扩缩容和镜像版本的升降级。
- ReplicaSet的资源清单文件:
apiVersion: apps/v1 # 版本号
kind: ReplicaSet # 类型
metadata: # 元数据
name: # rs名称
namespace: # 所属命名空间
labels: #标签
controller: rs
spec: # 详情描述
replicas: 3 # 副本数量
selector: # 选择器，通过它指定该控制器管理哪些pod
matchLabels: # Labels匹配规则
app: nginx-pod
matchExpressions: # Expressions匹配规则
- {key: app, operator: In, values: [nginx-pod]}
template: # 模板，当副本数量不足时，会根据下面的模板创建pod副本
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
在这里面的配置文件,需要重新了解的配置项就是spec下面几个选项:
- replicas:指定副本的数量,其实就是当前rs创建出来的数量,默认为1
- selector:选择器,它的作用是建立Pod控制器和Pod之间的关联关系, 采用的Label Selector机制在Pod模板上定义Label,在控制器上定义选择器,就可以表明当前控制器能管理哪些Pod了
- template:模板,就是当前控制器创建Pod所使用的的模板,里面其实就是之前学过的Pod的定义.
创建一个 ReplicaSet:
创建 pc-replicaset.yaml文件,内容如下.
cat <<EOF > pc.replicaset.yaml
apiVersion: apps/v1
kind: ReplicaSet
metadata:
name: pc-replicaset
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
EOF
#创建rs
[root@xnode1 ~]# kubectl create -f pc.replicaset.yaml
replicaset.apps/pc-replicaset created
#查看刚刚创建的rs:
#DESIRED:期望副本数量
#CURRENT:当前副本数量
#READY:已经准备好提供服务的副本数量
[root@xnode1 ~]# kubectl get rs -n dev pc-replicaset -o wide
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
pc-replicaset 3 3 3 62s nginx nginx:1.17.1 app=nginx-pod
#查看Pod
[root@xnode1 ~]# kubectl get pods -n dev -o wide | grep ^pc-replicaset
pc-replicaset-g5dgl 1/1 Running 0 5m21s 10.244.1.18 xnode2 <none> <none>
pc-replicaset-srnmz 1/1 Running 0 5m21s 10.244.1.19 xnode2 <none> <none>
pc-replicaset-vcjqt 1/1 Running 0 5m21s 10.244.1.17 xnode2 <none> <none>
#修改rs配置文件
[root@xnode1 ~]# kubectl edit rs -n dev pc-replicaset
15 spec:
16 replicas: 4 ==> 数量从3修改成4
17 selector:
18 matchLabels:
19 app: nginx-pod
20 template:
21 metadata:
22 creationTimestamp: null
23 labels:
24 app: nginx-pod
#查看rs和pod
[root@xnode1 ~]# kubectl get rs -n dev pc-replicaset -o wide
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
pc-replicaset 4 4 4 10m nginx nginx:1.17.1 app=nginx-pod
[root@xnode1 ~]# kubectl get pods -n dev -o wide | grep ^pc-replicaset
pc-replicaset-g5dgl 1/1 Running 0 10m 10.244.1.18 xnode2 <none> <none>
pc-replicaset-jgjtd 1/1 Running 0 23s 10.244.1.20 xnode2 <none> <none>
pc-replicaset-srnmz 1/1 Running 0 10m 10.244.1.19 xnode2 <none> <none>
pc-replicaset-vcjqt 1/1 Running 0 10m 10.244.1.17 xnode2 <none> <none>
#直接使用命令来实现
#使用`scale`命令实现扩缩容, 后面--replicas=n 直接指定目标数量即可
[root@xnode1 ~]# kubectl scale rs -n dev pc-replicaset --replicas=2
replicaset.extensions/pc-replicaset scaled
#再次查看rs和pod
[root@xnode1 ~]# kubectl get rs -n dev pc-replicaset -o wide
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
pc-replicaset 2 2 2 14m nginx nginx:1.17.1 app=nginx-pod
[root@xnode1 ~]# kubectl get pods -n dev -o wide | grep ^pc-replicaset
pc-replicaset-srnmz 1/1 Running 0 15m 10.244.1.19 xnode2 <none> <none>
pc-replicaset-vcjqt 1/1 Running 0 15m 10.244.1.17 xnode2 <none> <none>
#给Pod进行版本的升降级:修改配置文件
[root@xnode1 ~]# kubectl edit rs -n dev pc-replicaset
27 spec:
28 containers:
29 - image: nginx:1.17.1 ==>nginx:1.17.1修改成'nginx:1.17.2'
30 imagePullPolicy: IfNotPresent
31 name: nginx
32 resources: {}
33 terminationMessagePath: /dev/termination-log
34 terminationMessagePolicy: File
#查看版本:发现版本以及变更
[root@xnode1 ~]# kubectl get rs -n dev pc-replicaset -o wide
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
pc-replicaset 2 2 2 19m nginx nginx:1.17.2 app=nginx-pod
#使用命令对版本进行升降级:
[root@xnode1 ~]# kubectl set image rs pc-replicaset nginx=nginx:1.17.1 -n dev
replicaset.extensions/pc-replicaset image updated
[root@xnode1 ~]# kubectl get rs -n dev pc-replicaset -o wide
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
pc-replicaset 2 2 2 24m nginx nginx:1.17.1 app=nginx-pod
- 删除ReplicaSet:
#使用kubectl delete命令会删除此RS以及其它管理的Pod
#在kubernetes删除rs前,会将rs的replicasclear调整为0,等待所有的Pod被删除后,在执行RS对象的删除
kubectl delete rs pc-replicaset -n dev
#如果希望仅仅删除rs对象,保留(Pod),可以使用Kubectl delete命令添加--cascade=false选项,这样不推荐
kubectl delete rs pc-replicaset -n dev --cascade=false
#也可以使用yaml直接删除
[root@xnode1 ~]# kubectl delete -f pc.replicaset.yaml
replicaset.apps "pc-replicaset" deleted
（2）、Deployment （Deploy）
为了更好解决服务编排的问题,kubernetes在v:1.12.版本开始,引入了Deployment控制器,这个控制器并不是直接管理Pod,而是通过ReplicaSet来进行管理Pod
即:Deployment管理ReplicaSet,ReplicaSet管理Pod,所以Deployment比ReplicaSet功能更加强大.
Deployment 主要的功能有下面几个:
- 支持ReplicaSet的所有功能
- 支持发布的停止、继续
- 支持滚动升级和回滚版本
Deployment的资源清单文件：
apiVersion: apps/v1 # 版本号
kind: Deployment # 类型
metadata: # 元数据
name: # rs名称
namespace: # 所属命名空间
labels: #标签
controller: deploy
spec: # 详情描述
replicas: 3 # 副本数量
revisionHistoryLimit: 3 # 保留历史版本
paused: false # 暂停部署，默认是false
progressDeadlineSeconds: 600 # 部署超时时间（s），默认是600
strategy: # 策略
type: RollingUpdate # 滚动更新策略
rollingUpdate: # 滚动更新
maxSurge: 30% # 最大额外可以存在的副本数，可以为百分比，也可以为整数
maxUnavailable: 30% # 最大不可用状态的 Pod 的最大值，可以为百分比，也可以为整数
selector: # 选择器，通过它指定该控制器管理哪些pod
matchLabels: # Labels匹配规则
app: nginx-pod
matchExpressions: # Expressions匹配规则
- {key: app, operator: In, values: [nginx-pod]}
template: # 模板，当副本数量不足时，会根据下面的模板创建pod副本
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
- 创建Deployment:
cat <<EOF > pc-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: pc-deployment
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
EOF
#创建deployment
[root@xnode1 ~]# kubectl create -f pc-deployment.yaml
deployment.apps/pc-deployment created
#查看:
#UP-TO-DATE 最新版本的pod的数量
#AVAILABLE 当前可用的pod的数量
[root@xnode1 ~]# kubectl get deployment -n dev -o wide | grep ^pc-deployment
pc-deployment 3/3 3 3 28s nginx nginx:1.17.1 app=nginx-pod
#查看rs:发现rs的名称是在原来的deployment的名字后面添加了一个10位数的随机串
[root@xnode1 ~]# kubectl get rs -n dev -o wide
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
pc-deployment-77d96bb58b 3 3 3 5m5s nginx nginx:1.17.1 app=nginx-pod,pod-template-hash=77d96bb58b
扩缩容:
- 命令行
- 编辑配置文件
#使用命令行进行扩缩容
[root@xnode1 ~]# kubectl scale deployment pc-deployment --replicas=5 -n dev
deployment.extensions/pc-deployment scaled
[root@xnode1 ~]# kubectl get pod -n dev | grep ^pc-deployment
pc-deployment-77d96bb58b-8kf2l 1/1 Running 0 20m
pc-deployment-77d96bb58b-fkrbz 1/1 Running 0 20m
pc-deployment-77d96bb58b-gfw2x 1/1 Running 0 34s
pc-deployment-77d96bb58b-m84wl 1/1 Running 0 20m
pc-deployment-77d96bb58b-nbbbq 1/1 Running 0 34s
#使用修改配置文件的方式进行缩扩容
[root@xnode1 ~]# kubectl edit deployment pc-deployment -n dev
17 spec:
18 progressDeadlineSeconds: 600
19 replicas: 3 ==>修改为3
20 revisionHistoryLimit: 10
21 selector:
22 matchLabels:
23 app: nginx-pod
[root@xnode1 ~]# kubectl get pod -n dev
NAME READY STATUS RESTARTS AGE
pc-deployment-77d96bb58b-8f862 1/1 Running 0 4s
pc-deployment-77d96bb58b-8kf2l 1/1 Running 0 31m
pc-deployment-77d96bb58b-bkfn9 1/1 Running 0 4s
镜像更新:
Deployment 支持两种镜像更新的策略:
重建更新和滚动更新(默认),可以通过strategy选项进行配置.
strategy：指定新的Pod替换旧的Pod的策略， 支持两个属性：
type：指定策略类型，支持两种策略
Recreate：在创建出新的Pod之前会先杀掉所有已存在的Pod
RollingUpdate：滚动更新，就是杀死一部分，就启动一部分，在更新过程中，存在两个版本Pod
rollingUpdate：当type为RollingUpdate时生效，用于为RollingUpdate设置参数，支持两个属性：
maxUnavailable：用来指定在升级过程中不可用Pod的最大数量，默认为25%。
maxSurge： 用来指定在升级过程中可以超过期望的Pod的最大数量，默认为25%。
- 重建更新
- 编辑 pc-deployment.yaml在spec节点下添加更新策略
spec:
strategy: # 策略
type: Recreate # 重建更新
- 创建 deploy进行验证
[root@xnode1 ~]# cat <<EOF > pc-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: pc-deployment
namespace: dev
spec:
strategy: # 策略
type: Recreate # 重建更新
replicas: 3
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
EOF
#应用更新策略
[root@xnode1 ~]# kubectl apply -f pc-deployment.yaml
#执行
[root@xnode1 ~]# kubectl set image deploy pc-deployment nginx=nginx:1.17.2 -n dev
deployment.extensions/pc-deployment image updated
#查看更新状态
[root@xnode1 ~]# kubectl get pods -n dev -w
NAME READY STATUS RESTARTS AGE
pc-deployment-77d96bb58b-8f862 1/1 Running 0 4m36s
pc-deployment-77d96bb58b-8kf2l 1/1 Running 0 35m
pc-deployment-77d96bb58b-bkfn9 1/1 Running 0 4m36s
taint1-56b4468b94-tbt4z 1/1 Running 0 4d23h
taint2-88556f6-dhhgl 1/1 Running 0 4d23h
taint3-69b4df4bf4-gzf9l 1/1 Running 0 4d23h
^C[root@xnode1 ~]#
[root@xnode1 ~]# kubectl get pods -n dev -w | grep ^pc-deployment
pc-deployment-77d96bb58b-8f862 1/1 Running 0 4m52s
pc-deployment-77d96bb58b-8kf2l 1/1 Running 0 36m
pc-deployment-77d96bb58b-bkfn9 1/1 Running 0 4m52s
pc-deployment-77d96bb58b-8f862 1/1 Terminating 0 5m42s
pc-deployment-77d96bb58b-bkfn9 1/1 Terminating 0 5m42s
pc-deployment-77d96bb58b-8kf2l 1/1 Terminating 0 36m
pc-deployment-77d96bb58b-8kf2l 0/1 Terminating 0 37m
pc-deployment-77d96bb58b-bkfn9 0/1 Terminating 0 5m46s
pc-deployment-77d96bb58b-8f862 0/1 Terminating 0 5m46s
pc-deployment-77d96bb58b-8kf2l 0/1 Terminating 0 37m
pc-deployment-77d96bb58b-8kf2l 0/1 Terminating 0 37m
pc-deployment-77d96bb58b-8f862 0/1 Terminating 0 5m53s
pc-deployment-77d96bb58b-8f862 0/1 Terminating 0 5m53s
pc-deployment-77d96bb58b-bkfn9 0/1 Terminating 0 5m53s
pc-deployment-77d96bb58b-bkfn9 0/1 Terminating 0 5m53s
pc-deployment-54fd46c697-zq2l4 0/1 Pending 0 0s
pc-deployment-54fd46c697-zq2l4 0/1 Pending 0 0s
pc-deployment-54fd46c697-kjftb 0/1 Pending 0 0s
pc-deployment-54fd46c697-fjxv6 0/1 Pending 0 0s
pc-deployment-54fd46c697-kjftb 0/1 Pending 0 0s
pc-deployment-54fd46c697-fjxv6 0/1 Pending 0 0s
pc-deployment-54fd46c697-zq2l4 0/1 ContainerCreating 0 0s
pc-deployment-54fd46c697-kjftb 0/1 ContainerCreating 0 1s
pc-deployment-54fd46c697-fjxv6 0/1 ContainerCreating 0 1s
pc-deployment-54fd46c697-zq2l4 1/1 Running 0 29s
pc-deployment-54fd46c697-kjftb 1/1 Running 0 35s
pc-deployment-54fd46c697-fjxv6 1/1 Running 0 51s
- 滚动更新
01.编辑pc-deployment.yaml,在spec节点上添加更新策略
cat <<EOF > pc-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: pc-deployment
namespace: dev
spec:
strategy: # 策略
type: RollingUpdate # 滚动更新策略
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
replicas: 3
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
EOF
#应用更新策略
[root@xnode1 ~]# kubectl apply -f pc-deployment.yaml
deployment.apps/pc-deployment configured
#启动更新
[root@xnode1 ~]# kubectl set image deploy pc-deployment nginx=nginx:1.17.3 -n dev
deployment.extensions/pc-deployment image updated
#查看Pod状态
[root@xnode1 ~]# kubectl get pods -n dev -w | grep ^pc-deployment
pc-deployment-77d96bb58b-8w5rx 1/1 Running 0 60s
pc-deployment-77d96bb58b-kbp4h 1/1 Running 0 53s
pc-deployment-77d96bb58b-vvngp 1/1 Running 0 50s
pc-deployment-7975858c88-r55gc 0/1 Pending 0 0s
pc-deployment-7975858c88-r55gc 0/1 Pending 0 0s
pc-deployment-7975858c88-r55gc 0/1 ContainerCreating 0 0s
pc-deployment-7975858c88-r55gc 1/1 Running 0 10s
pc-deployment-77d96bb58b-vvngp 1/1 Terminating 0 97s
pc-deployment-7975858c88-bwht8 0/1 Pending 0 0s
pc-deployment-7975858c88-bwht8 0/1 Pending 0 0s
pc-deployment-7975858c88-bwht8 0/1 ContainerCreating 0 0s
pc-deployment-77d96bb58b-vvngp 0/1 Terminating 0 98s
pc-deployment-7975858c88-bwht8 1/1 Running 0 2s
pc-deployment-77d96bb58b-kbp4h 1/1 Terminating 0 102s
pc-deployment-7975858c88-rv5rp 0/1 Pending 0 0s
pc-deployment-77d96bb58b-vvngp 0/1 Terminating 0 99s
pc-deployment-7975858c88-rv5rp 0/1 Pending 0 0s
pc-deployment-77d96bb58b-vvngp 0/1 Terminating 0 99s
pc-deployment-7975858c88-rv5rp 0/1 ContainerCreating 0 1s
pc-deployment-77d96bb58b-kbp4h 0/1 Terminating 0 103s
pc-deployment-7975858c88-rv5rp 1/1 Running 0 2s
pc-deployment-77d96bb58b-8w5rx 1/1 Terminating 0 112s
pc-deployment-77d96bb58b-kbp4h 0/1 Terminating 0 105s
pc-deployment-77d96bb58b-kbp4h 0/1 Terminating 0 105s
pc-deployment-77d96bb58b-8w5rx 0/1 Terminating 0 112s
pc-deployment-77d96bb58b-8w5rx 0/1 Terminating 0 113s
pc-deployment-77d96bb58b-8w5rx 0/1 Terminating 0 114s
滚动更新的过程:
- 镜像更新中rs的变化
#查看rs,发现原来的rs的依旧存在,只是Pod数量变为了0,而后又新产生了一个rs,pod数量为4
#这就是deployment能够进行版本回退的奥妙所在
[root@xnode1 ~]# kubectl get rs -n dev | grep ^pc-deployment
pc-deployment-54fd46c697 0 0 0 128m
pc-deployment-77d96bb58b 0 0 0 165m
pc-deployment-7975858c88 3 3 3 115m
版本回退:
deployment支持版本升级过程中的暂停,继续功能以及版本回退等诸多功能,下面具体来看.
kubectl rollout: 版本升级相关功能,支持下面的选项:
- status 显示当前升级状态
- history 显示升级历史记录
- pause 暂停版本升级过程
- resume 继续以及暂停的版本升级过程
- restart 重启版本升级过程
- undo 回滚到上一级版本(可以使用 --to-version回滚到指定版本)
#查看升级状态
[root@xnode1 ~]# kubectl rollout status deploy -n dev pc-deployment
deployment "pc-deployment" successfully rolled out
#显示升级历史记录
[root@xnode1 ~]# kubectl rollout history deploy -n dev pc-deployment
deployment.extensions/pc-deployment
REVISION CHANGE-CAUSE
1 kubectl create --filename=pc-deployment.yaml --record=true
2 kubectl create --filename=pc-deployment.yaml --record=true
#版本回退
#查看当前版本
[root@xnode1 ~]# kubectl get deployments -n dev -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
pc-deployment 3/3 3 3 16m nginx nginx:1.17.2 app=nginx-pod
[root@xnode1 ~]# kubectl rollout undo deployment pc-deployment --to-revision=1 -n dev
deployment.extensions/pc-deployment rolled back
[root@xnode1 ~]# kubectl get deployments -n dev -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
pc-deployment 3/3 3 3 19m nginx nginx:1.17.1 app=nginx-pod
[root@xnode1 ~]# kubectl get deployment,rs -n dev -o wide | grep pc-deployment
deployment.extensions/pc-deployment 3/3 3 3 22m nginx nginx:1.17.1 app=nginx-pod
replicaset.extensions/pc-deployment-54fd46c697 0 0 0 19m nginx nginx:1.17.2 app=nginx-pod,pod-template-hash=54fd46c697
replicaset.extensions/pc-deployment-77d96bb58b 3 3 3 22m nginx nginx:1.17.1 app=nginx-pod,pod-template-hash=77d96bb58b
#查看回退历史记录
[root@xnode1 ~]# kubectl rollout history deployment pc-deployment -n dev
deployment.extensions/pc-deployment
REVISION CHANGE-CAUSE
2 kubectl create --filename=pc-deployment.yaml --record=true
3 kubectl create --filename=pc-deployment.yaml --record=true
金丝雀:
Deployment控制器支持控制更新过程中的控制，如“暂停(pause)”或“继续(resume)”更新操作。
比如有一批新的Pod资源创建完成后立即暂停更新过程，此时，仅存在一部分新版本的应用，主体部分还是旧的版本。然后，再筛选一小部分的用户请求路由到新版本的Pod应用，继续观察能否稳定地按期望的方式运行。确定没问题之后再继续完成余下的Pod资源滚动更新，否则立即回滚更新操作。这就是所谓的金丝雀发布。
#更新deployment,并配置暂停deployment
[root@xnode1 ~]# kubectl set image deploy pc-deployment nginx=nginx:1.17.4 -n dev && kubectl rollout pause deployment pc-deployment -n dev
deployment.extensions/pc-deployment image updated
deployment.extensions/pc-deployment paused
#查看此时的deployment更新状态
[root@xnode1 ~]# kubectl rollout status deployment pc-deployment -n dev
Waiting for deployment "pc-deployment" rollout to finish: 1 out of 3 new replicas have been updated...
#监控更新的过程,可以看到一部分的pod以及更新了一部分,但是没有按照预期的状态去删除一个旧的资源,因为我们上面在更新后面使用了pause暂停命令
[root@xnode1 ~]# kubectl get rs -n dev
NAME DESIRED CURRENT READY AGE
pc-deployment-54fd46c697 0 0 0 30m
pc-deployment-69ff4bbfcf 1 1 1 58s
pc-deployment-77d96bb58b 3 3 3 33m
#确认pod没问题了,继续更新
[root@xnode1 ~]# kubectl rollout resume deployment pc-deployment -n dev
deployment.extensions/pc-deployment resumed
[root@xnode1 ~]# kubectl rollout status deployment pc-deployment -n dev
Waiting for deployment "pc-deployment" rollout to finish: 1 out of 3 new replicas have been updated...
Waiting for deployment spec update to be observed...
Waiting for deployment spec update to be observed...
Waiting for deployment "pc-deployment" rollout to finish: 1 out of 3 new replicas have been updated...
Waiting for deployment "pc-deployment" rollout to finish: 1 out of 3 new replicas have been updated...
Waiting for deployment "pc-deployment" rollout to finish: 2 out of 3 new replicas have been updated...
Waiting for deployment "pc-deployment" rollout to finish: 2 out of 3 new replicas have been updated...
Waiting for deployment "pc-deployment" rollout to finish: 2 out of 3 new replicas have been updated...
Waiting for deployment "pc-deployment" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "pc-deployment" rollout to finish: 1 old replicas are pending termination...
deployment "pc-deployment" successfully rolled out
（3）、Horizontal Pod Autoscaler(HPA)
通过前面的课程学习,我们以及可以实现通过手工
kubectl scale命令实现pod扩容或缩容,但是这并不符合kubernetes的定位目标-自动化.kubernetes期望的是通过监测pod的使用情况,实现pod数量的自动调整,于是就产生了Horizontal Pod Autoscaler(HPA)HPA原理:
HPA可以获得每个Pod的利用率,然后HPA中定义的指标进行对比,同时计算出需要伸缩的具体值,最后实现Pod数量的调整,其实HPA与之前的Deployment一样,也属于一种kubernetes的资源对象,它通过追踪分析RC控制的所有目标Pod负载变化情况,来确定是否需要针对性的调整目标Pod的副本数,
1、安装metrics-server
- metrics-server可以用来收集集群中的资源使用情况
#安装git
[root@xnode1 ~]# yum install git -y
[root@xnode1 ~]# git clone -b v0.3.6 https://github.com/kubernetes-incubator/metrics-server
#修改配置文件
[root@xnode1 ~]# vim /root/metrics-server/deploy/1.8+/metrics-server-deployment.yaml
24 spec:
25 hostNetwork: true ==>添加
26 serviceAccountName: metrics-server
27 volumes:
28 # mount in tmp so we can safely use from-scratch images and/or read-only containers
29 - name: tmp-dir
30 emptyDir: {}
31 containers:
32 - name: metrics-server
33 image: registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-server-amd64:v0.3.6 ==>修改
34 imagePullPolicy: Always
35 args: ==>添加
36 - --kubelet-insecure-tls ==>添加
37 - --kubelet-preferred-address-types=InternalIP,Hostname,InternalDNS,ExternalDNS,ExternalIP==>添加
38 volumeMounts:
39 - name: tmp-dir
40 mountPath: /tmp
[root@xnode1 ~]# cd /root/metrics-server/deploy/1.8+/
[root@xnode1 1.8+]# kubectl apply -f ./
clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created
clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created
rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created
apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created
serviceaccount/metrics-server created
deployment.apps/metrics-server created
service/metrics-server created
clusterrole.rbac.authorization.k8s.io/system:metrics-server created
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created
#查看
[root@xnode1 1.8+]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-8686dcc4fd-f9pqv 1/1 Running 4 6d
coredns-8686dcc4fd-mtcmn 1/1 Running 7 6d
etcd-xnode1 1/1 Running 4 6d
kube-apiserver-xnode1 1/1 Running 4 6d
kube-controller-manager-xnode1 1/1 Running 4 6d
kube-flannel-ds-amd64-67htn 1/1 Running 3 6d
kube-flannel-ds-amd64-frh8l 1/1 Running 4 6d
kube-proxy-mvgrq 1/1 Running 2 5d18h
kube-proxy-ptlm8 1/1 Running 4 5d20h
kube-proxy-rjgfs 1/1 Running 2 5d20h
kube-scheduler-xnode1 1/1 Running 5 6d
metrics-server-55f688dd5f-wgjjc 1/1 Running 0 32s
#查看资源的使用情况
[root@xnode1 1.8+]# kubectl top node
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
xnode1 166m 8% 1619Mi 42%
xnode2 39m 3% 748Mi 40%
xnode3 27m 2% 351Mi 18%
[root@xnode1 1.8+]# kubectl top pod -n kube-system
NAME CPU(cores) MEMORY(bytes)
coredns-8686dcc4fd-f9pqv 3m 15Mi
coredns-8686dcc4fd-mtcmn 3m 17Mi
etcd-xnode1 20m 67Mi
kube-apiserver-xnode1 28m 236Mi
kube-controller-manager-xnode1 15m 42Mi
kube-flannel-ds-amd64-67htn 2m 19Mi
kube-flannel-ds-amd64-frh8l 3m 12Mi
kube-proxy-mvgrq 4m 17Mi
kube-proxy-ptlm8 1m 18Mi
kube-proxy-rjgfs 1m 20Mi
kube-scheduler-xnode1 3m 16Mi
metrics-server-55f688dd5f-wgjjc 1m 12Mi
2、准备deployment和service
#创建deployment
[root@xnode1 ~]# kubectl run nginx --image=nginx:1.17.1 --requests=cpu=100m -n dev
kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
deployment.apps/nginx created
[root@xnode1 ~]# kubectl get deployment,pod -n dev
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.extensions/nginx 1/1 1 1 60s
NAME READY STATUS RESTARTS AGE
pod/nginx-84c556bc5c-f6xvm 1/1 Running 0 60s
#创建service
[root@xnode1 ~]# kubectl expose deployment nginx --type=NodePort --port=80 -n dev
service/nginx exposed
3、部署HPA
#创建pc-hpa.yaml
cat <<EOF > pc-hpa.yaml
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
name: pc-hpa
namespace: dev
spec:
minReplicas: 1 #最小pod数量
maxReplicas: 10 #最大pod数量
targetCPUUtilizationPercentage: 3 # CPU使用率指标
scaleTargetRef: # 指定要控制的nginx信息
apiVersion: apps/v1
kind: Deployment
name: nginx
EOF
#创建并查看
[root@xnode1 ~]# kubectl create -f pc-hpa.yaml
horizontalpodautoscaler.autoscaling/pc-hpa created
[root@xnode1 ~]# kubectl get hpa -n dev
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
pc-hpa Deployment/nginx 0%/3% 1 10 1 29s
4、测试：
- 使用测压工具对service地址10.0.0.30:32647进行测试
- 通过控制台查看hpa和pod的变化
#hpa变化
[root@xnode1 ~]# kubectl get hpa -n dev -w
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
pc-hpa Deployment/nginx 0%/3% 1 10 1 4m11s
pc-hpa Deployment/nginx 0%/3% 1 10 1 5m19s
pc-hpa Deployment/nginx 22%/3% 1 10 1 6m50s
pc-hpa Deployment/nginx 22%/3% 1 10 4 7m5s
pc-hpa Deployment/nginx 22%/3% 1 10 8 7m21s
pc-hpa Deployment/nginx 6%/3% 1 10 8 7m51s
pc-hpa Deployment/nginx 0%/3% 1 10 8 9m6s
pc-hpa Deployment/nginx 0%/3% 1 10 8 13m
pc-hpa Deployment/nginx 0%/3% 1 10 1 14m
#deployment变化
[root@xnode1 ~]# kubectl get deployment -n dev -w
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 1/1 1 1 11m
nginx 1/4 1 1 13m
nginx 1/4 1 1 13m
nginx 1/4 1 1 13m
nginx 1/4 4 1 13m
nginx 1/8 4 1 14m
nginx 1/8 4 1 14m
nginx 1/8 4 1 14m
nginx 1/8 8 1 14m
nginx 2/8 8 2 14m
nginx 3/8 8 3 14m
nginx 4/8 8 4 14m
nginx 5/8 8 5 14m
nginx 6/8 8 6 14m
nginx 7/8 8 7 14m
nginx 8/8 8 8 15m
nginx 8/1 8 8 20m
nginx 8/1 8 8 20m
nginx 1/1 1 1 20m
#pod变化
[root@xnode1 ~]# kubectl get pods -n dev -w
NAME READY STATUS RESTARTS AGE
nginx-7df9756ccc-bh8dr 1/1 Running 0 11m
nginx-7df9756ccc-cpgrv 0/1 Pending 0 0s
nginx-7df9756ccc-8zhwk 0/1 Pending 0 0s
nginx-7df9756ccc-rr9bn 0/1 Pending 0 0s
nginx-7df9756ccc-cpgrv 0/1 ContainerCreating 0 0s
nginx-7df9756ccc-8zhwk 0/1 ContainerCreating 0 0s
nginx-7df9756ccc-rr9bn 0/1 ContainerCreating 0 0s
nginx-7df9756ccc-m9gsj 0/1 Pending 0 0s
nginx-7df9756ccc-g56qb 0/1 Pending 0 0s
nginx-7df9756ccc-sl9c6 0/1 Pending 0 0s
nginx-7df9756ccc-fgst7 0/1 Pending 0 0s
nginx-7df9756ccc-g56qb 0/1 ContainerCreating 0 0s
nginx-7df9756ccc-m9gsj 0/1 ContainerCreating 0 0s
nginx-7df9756ccc-sl9c6 0/1 ContainerCreating 0 0s
nginx-7df9756ccc-fgst7 0/1 ContainerCreating 0 0s
nginx-7df9756ccc-8zhwk 1/1 Running 0 19s
nginx-7df9756ccc-rr9bn 1/1 Running 0 30s
nginx-7df9756ccc-m9gsj 1/1 Running 0 21s
nginx-7df9756ccc-cpgrv 1/1 Running 0 47s
nginx-7df9756ccc-sl9c6 1/1 Running 0 33s
nginx-7df9756ccc-g56qb 1/1 Running 0 48s
nginx-7df9756ccc-fgst7 1/1 Running 0 66s
nginx-7df9756ccc-fgst7 1/1 Terminating 0 6m50s
nginx-7df9756ccc-8zhwk 1/1 Terminating 0 7m5s
nginx-7df9756ccc-cpgrv 1/1 Terminating 0 7m5s
nginx-7df9756ccc-g56qb 1/1 Terminating 0 6m50s
nginx-7df9756ccc-rr9bn 1/1 Terminating 0 7m5s
nginx-7df9756ccc-m9gsj 1/1 Terminating 0 6m50s
nginx-7df9756ccc-sl9c6 1/1 Terminating 0 6m50s
（4）、DaemonSet（DS）
DaemonSet:
DaemonSet 类型的控制器可以保证集群中的每一台(或指定)节点上都运行一个副本,一般是用于日志收集、节点控制等场景。也就是说，如果一个pod提供的功能是节点级别的（每个节点都需要且只需要一个），那么这类pod就适合DaemonSet类型的控制器创建。
DaemonSet控制器的特点 :
- 每当向集群中添加一个节点时,指定的pod副本也将添加到该节点上
- 每当节点从集群中移除时,pod也就被垃圾回收了
1、查看一下DaemonSet的资源清单文件
apiVersion: apps/v1 # 版本号
kind: DaemonSet # 类型
metadata: # 元数据
name: # rs名称
namespace: # 所属命名空间
labels: #标签
controller: daemonset
spec: # 详情描述
revisionHistoryLimit: 3 # 保留历史版本
updateStrategy: # 更新策略
type: RollingUpdate # 滚动更新策略
rollingUpdate: # 滚动更新
maxUnavailable: 1 # 最大不可用状态的 Pod 的最大值，可以为百分比，也可以为整数
selector: # 选择器，通过它指定该控制器管理哪些pod
matchLabels: # Labels匹配规则
app: nginx-pod
matchExpressions: # Expressions匹配规则
- {key: app, operator: In, values: [nginx-pod]}
template: # 模板，当副本数量不足时，会根据下面的模板创建pod副本
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
2、创建pc-daemonset.yaml
cat <<EOF > pc-daemonset.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: pc-daemonset
namespace: dev
spec:
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
EOF
3、查看
[root@xnode1 ~]# kubectl create -f pc-daemonset.yaml
daemonset.apps/pc-daemonset created
[root@xnode1 ~]# kubectl get daemonset -n dev -o wide
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE CONTAINERS IMAGES SELECTOR
pc-daemonset 2 2 2 2 2 <none> 14s nginx nginx:1.17.1 app=nginx-pod
[root@xnode1 ~]# kubectl get pods -n dev -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pc-daemonset-pcdt6 1/1 Running 0 37s 10.244.2.9 xnode3 <none> <none>
pc-daemonset-zmdh8 1/1 Running 0 37s 10.244.1.61 xnode2 <none> <none>
4、删除daemonset
[root@xnode1 ~]# kubectl delete -f pc-daemonset.yaml
daemonset.apps "pc-daemonset" deleted
（5）、Job
Job:
Job主要用于负责 批量处理短暂的 一次性任务,Job的特点:
- 当 Job创建的Pod执行成功结束时,Job将记录成功结束的pod数量
- 当成功结束的Pod达到指定的数量时,Job将完成执行
1、Job资源清单文件
apiVersion: batch/v1 # 版本号
kind: Job # 类型
metadata: # 元数据
name: # rs名称
namespace: # 所属命名空间
labels: #标签
controller: job
spec: # 详情描述
completions: 1 # 指定job需要成功运行Pods的次数。默认值: 1
parallelism: 1 # 指定job在任一时刻应该并发运行Pods的数量。默认值: 1
activeDeadlineSeconds: 30 # 指定job可运行的时间期限，超过时间还未结束，系统将会尝试进行终止。
backoffLimit: 6 # 指定job失败后进行重试的次数。默认是6
manualSelector: true # 是否可以使用selector选择器选择pod，默认是false
selector: # 选择器，通过它指定该控制器管理哪些pod
matchLabels: # Labels匹配规则
app: counter-pod
matchExpressions: # Expressions匹配规则
- {key: app, operator: In, values: [counter-pod]}
template: # 模板，当副本数量不足时，会根据下面的模板创建pod副本
metadata:
labels:
app: counter-pod
spec:
restartPolicy: Never # 重启策略只能设置为Never或者OnFailure
containers:
- name: counter
image: busybox:1.30
command: ["bin/sh","-c","for i in 9 8 7 6 5 4 3 2 1; do echo $i;sleep 2;done"]
重启策略说明:
如果指定为OnFailure，则job会在pod出现故障时重启容器，而不是创建pod，failed次数不变
如果指定为Never，则job会在pod出现故障时创建新的pod，并且故障pod不会消失，也不会重启，failed次数加1
如果指定为Always的话，就意味着一直重启，意味着job任务会重复去执行了，当然不对，所以不能设置为Always
2、创建pc-job.yaml
cat <<EOF > pc-job.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: pc-job
namespace: dev
spec:
manualSelector: true
selector:
matchLabels:
app: counter-pod
template:
metadata:
labels:
app: counter-pod
spec:
restartPolicy: Never
containers:
- name: counter
image: busybox:1.30
command: ["bin/sh","-c","for i in 9 8 7 6 5 4 3 2 1; do echo $i;sleep 3;done"]
EOF
3、创建并查看
#创建Job
[root@xnode1 ~]# kubectl create -f pc-job.yaml
job.batch/pc-job created
#监控Job
[root@xnode1 ~]# kubectl get job -n dev -o wide -w
NAME COMPLETIONS DURATION AGE CONTAINERS IMAGES SELECTOR
pc-job 0/1 0s counter busybox:1.30 app=counter-pod
pc-job 0/1 0s 0s counter busybox:1.30 app=counter-pod
pc-job 1/1 29s 29s counter busybox:1.30 app=counter-pod
#监控Pod
[root@xnode1 ~]# kubectl get pod -n dev -o wide -w
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pc-job-8j5kz 0/1 Pending 0 0s <none> <none> <none> <none>
pc-job-8j5kz 0/1 Pending 0 0s <none> xnode2 <none> <none>
pc-job-8j5kz 0/1 ContainerCreating 0 0s <none> xnode2 <none> <none>
pc-job-8j5kz 1/1 Running 0 2s 10.244.1.63 xnode2 <none> <none>
pc-job-8j5kz 0/1 Completed 0 29s 10.244.1.63 xnode2 <none> <none>
#修改pc-job.yaml
cat <<EOF > pc-job.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: pc-job
namespace: dev
spec:
manualSelector: true
completions: 6 # 指定job需要成功运行Pods的次数。默认值: 1 ==>添加
parallelism: 3 # 指定job在任一时刻应该并发运行Pods的数量。默认值: 1 ==>添加
selector:
matchLabels:
app: counter-pod
template:
metadata:
labels:
app: counter-pod
spec:
restartPolicy: Never
containers:
- name: counter
image: busybox:1.30
command: ["bin/sh","-c","for i in 9 8 7 6 5 4 3 2 1; do echo ;sleep 3;done"]
EOF
#再次观察Job和Pod
#发现Job每次运行3个Pod,总共执行了6个Pod
[root@xnode1 ~]# kubectl get job -n dev -o wide -w
NAME COMPLETIONS DURATION AGE CONTAINERS IMAGES SELECTOR
pc-job 0/6 0s counter busybox:1.30 app=counter-pod
pc-job 0/6 0s 0s counter busybox:1.30 app=counter-pod
pc-job 1/6 31s 31s counter busybox:1.30 app=counter-pod
pc-job 2/6 32s 32s counter busybox:1.30 app=counter-pod
pc-job 3/6 32s 32s counter busybox:1.30 app=counter-pod
pc-job 4/6 60s 60s counter busybox:1.30 app=counter-pod
pc-job 5/6 62s 62s counter busybox:1.30 app=counter-pod
pc-job 6/6 62s 62s counter busybox:1.30 app=counter-pod
[root@xnode1 ~]# kubectl get pod -n dev -o wide -w
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pc-job-gxs4q 0/1 Pending 0 0s <none> <none> <none> <none>
pc-job-gxs4q 0/1 Pending 0 0s <none> xnode2 <none> <none>
pc-job-cz7cw 0/1 Pending 0 0s <none> <none> <none> <none>
pc-job-9z6hk 0/1 Pending 0 0s <none> <none> <none> <none>
pc-job-cz7cw 0/1 Pending 0 0s <none> xnode2 <none> <none>
pc-job-9z6hk 0/1 Pending 0 0s <none> xnode2 <none> <none>
pc-job-gxs4q 0/1 ContainerCreating 0 0s <none> xnode2 <none> <none>
pc-job-cz7cw 0/1 ContainerCreating 0 0s <none> xnode2 <none> <none>
pc-job-9z6hk 0/1 ContainerCreating 0 0s <none> xnode2 <none> <none>
pc-job-cz7cw 1/1 Running 0 5s 10.244.1.65 xnode2 <none> <none>
pc-job-gxs4q 1/1 Running 0 5s 10.244.1.64 xnode2 <none> <none>
pc-job-9z6hk 1/1 Running 0 5s 10.244.1.66 xnode2 <none> <none>
pc-job-gxs4q 0/1 Completed 0 31s 10.244.1.64 xnode2 <none> <none>
pc-job-5xp4q 0/1 Pending 0 0s <none> <none> <none> <none>
pc-job-5xp4q 0/1 Pending 0 0s <none> xnode2 <none> <none>
pc-job-5xp4q 0/1 ContainerCreating 0 0s <none> xnode2 <none> <none>
pc-job-9z6hk 0/1 Completed 0 32s 10.244.1.66 xnode2 <none> <none>
pc-job-xqd8m 0/1 Pending 0 0s <none> <none> <none> <none>
pc-job-xqd8m 0/1 Pending 0 0s <none> xnode2 <none> <none>
pc-job-xqd8m 0/1 ContainerCreating 0 0s <none> xnode2 <none> <none>
pc-job-cz7cw 0/1 Completed 0 32s 10.244.1.65 xnode2 <none> <none>
pc-job-v7msn 0/1 Pending 0 0s <none> <none> <none> <none>
pc-job-v7msn 0/1 Pending 0 0s <none> xnode2 <none> <none>
pc-job-v7msn 0/1 ContainerCreating 0 0s <none> xnode2 <none> <none>
pc-job-5xp4q 1/1 Running 0 2s 10.244.1.67 xnode2 <none> <none>
pc-job-v7msn 1/1 Running 0 3s 10.244.1.69 xnode2 <none> <none>
pc-job-xqd8m 1/1 Running 0 3s 10.244.1.68 xnode2 <none> <none>
pc-job-5xp4q 0/1 Completed 0 29s 10.244.1.67 xnode2 <none> <none>
pc-job-xqd8m 0/1 Completed 0 30s 10.244.1.68 xnode2 <none> <none>
pc-job-v7msn 0/1 Completed 0 30s 10.244.1.69 xnode2 <none> <none>
4、End 删除pc-job
[root@xnode1 ~]# kubectl delete -f pc-job.yaml
job.batch "pc-job" deleted
（6）、CrontabJob（CJ）
CronJob控制器:
以Job控制器资源为管控对象,并借助它管理oid资源对象,Job控制器定义的作业任务在其控制器资源创建之后便会立即执行,但CronJob可以以类似于Linux操作系统的周期性任务作业计划的方式控制其运行时间点及重复运行的方式.也就是说，CronJob可以在特定的时间点(反复的)去运行job任务。
1、CronJob资源清单文件：
apiVersion: batch/v1beta1 # 版本号
kind: CronJob # 类型
metadata: # 元数据
name: # rs名称
namespace: # 所属命名空间
labels: #标签
controller: cronjob
spec: # 详情描述
schedule: # cron格式的作业调度运行时间点,用于控制任务在什么时间执行
concurrencyPolicy: # 并发执行策略，用于定义前一次作业运行尚未完成时是否以及如何运行后一次的作业
failedJobHistoryLimit: # 为失败的任务执行保留的历史记录数，默认为1
successfulJobHistoryLimit: # 为成功的任务执行保留的历史记录数，默认为3
startingDeadlineSeconds: # 启动作业错误的超时时长
jobTemplate: # job控制器模板，用于为cronjob控制器生成job对象;下面其实就是job的定义
metadata:
spec:
completions: 1
parallelism: 1
activeDeadlineSeconds: 30
backoffLimit: 6
manualSelector: true
selector:
matchLabels:
app: counter-pod
matchExpressions: 规则
- {key: app, operator: In, values: [counter-pod]}
template:
metadata:
labels:
app: counter-pod
spec:
restartPolicy: Never
containers:
- name: counter
image: busybox:1.30
command: ["bin/sh","-c","for i in 9 8 7 6 5 4 3 2 1; do echo $i;sleep 20;done"]
需要重点解释的几个选项：
schedule: cron表达式，用于指定任务的执行时间
*/1 * * * *
<分钟> <小时> <日> <月份> <星期>
分钟 值从 0 到 59.
小时 值从 0 到 23.
日 值从 1 到 31.
月 值从 1 到 12.
星期 值从 0 到 6, 0 代表星期日
多个时间可以用逗号隔开； 范围可以用连字符给出；*可以作为通配符； /表示每...
concurrencyPolicy:
Allow: 允许Jobs并发运行(默认)
Forbid: 禁止并发运行，如果上一次运行尚未完成，则跳过下一次运行
Replace: 替换，取消当前正在运行的作业并用新作业替换它
2、创建一个pc-cronjob.yaml
cat <<EOF > pc-cronjob.yaml
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: pc-cronjob
namespace: dev
labels:
controller: cronjob
spec:
schedule: "*/1 * * * *"
jobTemplate:
metadata:
spec:
template:
spec:
restartPolicy: Never
containers:
- name: counter
image: busybox:1.30
command: ["bin/sh","-c","for i in 9 8 7 6 5 4 3 2 1; do echo $i;sleep 3;done"]
EOF
3、创建并查看
可以看到cronjob每隔30s就执行一次任务,pod也是30s就新建一次
#创建cronjob
[root@xnode1 ~]# kubectl create -f pc-cronjob.yaml
cronjob.batch/pc-cronjob created
#查看cronjob
[root@xnode1 ~]# kubectl get cronjob -n dev -w
NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE
pc-cronjob */1 * * * * False 0 <none> 0s
pc-cronjob */1 * * * * False 1 6s 42s
pc-cronjob */1 * * * * False 0 36s 72s
pc-cronjob */1 * * * * False 1 6s 102s
pc-cronjob */1 * * * * False 0 36s 2m12s
#查看job
[root@xnode1 ~]# kubectl get job -n dev -w
NAME COMPLETIONS DURATION AGE
pc-cronjob-1661247480 0/1 0s
pc-cronjob-1661247480 0/1 0s 0s
pc-cronjob-1661247480 1/1 30s 30s
pc-cronjob-1661247540 0/1 0s
pc-cronjob-1661247540 0/1 0s 0s
pc-cronjob-1661247540 1/1 28s 28s
#查看pod
[root@xnode1 ~]# kubectl get pod -n dev -w
NAME READY STATUS RESTARTS AGE
pc-cronjob-1661247480-lgtrk 1/1 Running 0 20s
pc-cronjob-1661247480-lgtrk 0/1 Completed 0 30s
pc-cronjob-1661247540-s5xkj 0/1 Pending 0 0s
pc-cronjob-1661247540-s5xkj 0/1 Pending 0 0s
pc-cronjob-1661247540-s5xkj 0/1 ContainerCreating 0 0s
pc-cronjob-1661247540-s5xkj 1/1 Running 0 2s
pc-cronjob-1661247540-s5xkj 0/1 Completed 0 28s
#删除cronjob
[root@xnode1 ~]# kubectl delete -f pc-cronjob.yaml
cronjob.batch "pc-cronjob" deleted
十二、Service详解
Service介绍:
kubernetes中,pod是应用程序的载体,我们通过对pod的ip访问应用程序,但是pod的ip地址不是固定的,也就意味着不方便采用相同的pod得到ip对服务进行访问.
为了解决这个问题,kubernetes提供了Service资源,Service会对提供同一个服务的多个pod进行聚合,并且提供一个同一的入口地址.通过访问Services的入口地址就能访问到后面的Pod服务
Service在很多情况下只是一个概念,真正起作用的其实是 Kube-Proxy服务进程,每个Node节点上都运行着一个 Kube-Proxy服务进程,当创建Services的时候会通过 API-Server向 Etcd写入创建的service的信息里,而 Kube-Proxy 会基于监听的机制发现这种Service的变动,然后 它会将最新的Service信息转换成对应的访问规则.
#10.244.0.29:8443是service提供的访问入口
#当访问这个入口的时候,可以发现后面有3个pod的服务在等待调用
# kube-proxy会基于rr（轮询）的策略，将请求分发到其中一个pod上去
# 这个规则会同时在集群内的所有节点上都生成，所以在任何一个节点上访问都可以。
[root@xnode1 ~]# ipvsadm -Ln | grep 10.244
-> 10.244.0.29:8443 Masq 1 0 0
-> 10.244.0.29:8443 Masq 1 0 0
-> 10.244.0.29:8443 Masq 1 0 0
-> 10.244.0.29:8443 Masq 1 0 0
-> 10.244.0.29:8443 Masq 1 0 0
-> 10.244.0.29:8443 Masq 1 0 0
-> 10.244.0.29:8443 Masq 1 0 0
-> 10.244.0.29:8443 Masq 1 0 0
-> 10.244.0.27:53 Masq 1 0 0
-> 10.244.0.30:53 Masq 1 0 0
-> 10.244.0.27:9153 Masq 1 0 0
-> 10.244.0.30:9153 Masq 1 0 0
-> 10.244.0.29:8443 Masq 1 0 0
-> 10.244.0.28:8000 Masq 1 0 0
TCP 10.244.0.0:32429 rr
-> 10.244.0.29:8443 Masq 1 0 0
TCP 10.244.0.0:32647 rr
TCP 10.244.0.1:32429 rr
-> 10.244.0.29:8443 Masq 1 0 0
TCP 10.244.0.1:32647 rr
-> 10.244.0.29:8443 Masq 1 0 0
-> 10.244.0.27:53 Masq 1 0 0
-> 10.244.0.30:53 Masq 1 0 0
（1）、Kube-Proxy的三种模式
Kube-Proxy目前支持三种模式:
在这个模式下Kube-Proxy会为每一个Service创建一个监听端口,发向Cluster Ip的请求被Iptables规则重定向到 Kube-proxy监听端口上, Kube-proxy根据 LB算法选择一个提供服务的Pod并和其建立连接,以将请求转发到Pod上.该模式下,Kube-proxy充当了一个四层负载均衡器的角色.由于 Kube-proxy运行在userspace中,在进行转发处理时会增加内核和用户之间的数据拷贝,虽然比较稳定,但是效率比较低.
iptables模式:
iptables模式下,kube-proxy为service后端的每个pod创建对应的iptables规则,直接向cluster IP的请求重定向到一个Pod IP.该模式下Kube-Proxy不承担四层负载均衡的角色,只负责创建iptables规则.该模式的优点是较 userspace模式效率更高,但不能提供灵活的LB策略,当后端Pod也不可用时也无法进行重试.
ipvs模式:
ipvs模式和 iptables类似, kube-proxy监控pod的变化并创建相应的 ipvs规则,ipvs
相对 iptables转发规则效率更高,除此之外,ipvs支持更多的 Lb算法.
（2）、开启Ipvs
开启ipvs:
①.修改kube-proxy配置文件
②.重启kube-proxy
[root@xnode1 ~]# kubectl edit cm kube-proxy -n kube-system
32 ipvs:
33 excludeCIDRs: null
34 minSyncPeriod: 0s
35 scheduler: ""
36 syncPeriod: 30s
37 kind: KubeProxyConfiguration
38 metricsBindAddress: 127.0.0.1:10249
39 mode: "ipvs" ==>添加ipvs
40 nodePortAddresses: null
41 oomScoreAdj: -999
42 portRange: ""
43 resourceContainer: /kube-proxy
44 udpIdleTimeout: 250ms
[root@xnode1 ~]# kubectl get pod -n kube-system | grep kube-proxy | awk '{system("kubectl delete pod "$1" -n kube-system")}'
pod "kube-proxy-q7z99" deleted
pod "kube-proxy-vw9wf" deleted
测试ipvs:
①.由于已经通过了configmap修改了kube-proxy的配置,素有后期增加了节点会直接使用ipvs模式,我们这里查看一下日志
#查看一下kube-proxy容器的详细信息
[root@xnode1 ~]# kubectl describe pod -n kube-system kube-proxy
Name: kube-proxy-ptlm8
Namespace: kube-system
Priority: 2000001000
PriorityClassName: system-node-critical
Node: xnode1/10.0.0.30
Start Time: Wed, 17 Aug 2022 03:45:04 -0400
Labels: controller-revision-hash=5f46cbf776
k8s-app=kube-proxy
pod-template-generation=1
Annotations: <none>
Status: Running
IP: 10.0.0.30
Controlled By: DaemonSet/kube-proxy
Containers:
kube-proxy:
Container ID: docker://b7fc4a3b55063900c62415803b30097342e29a2dadd26277c7aac238cb49de55
Image: registry.aliyuncs.com/google_containers/kube-proxy:v1.14.1
Image ID: docker-pullable://registry.aliyuncs.com/google_containers/kube-proxy@sha256:44af2833c6cbd9a7fc2e9d2f5244a39dfd2e31ad91bf9d4b7d810678db738ee9
Port: <none>
Host Port: <none>
#查看kube-proxy-ptlm8这个pod的日志信息
[root@xnode1 ~]# kubectl logs kube-proxy-ptlm8 -n kube-system
I0817 07:45:05.902030 1 server_others.go:177] Using ipvs Proxier. ==>可以看到这里已经在使用ipvs
W0817 07:45:05.902743 1 proxier.go:381] IPVS scheduler not specified, use rr by default
I0817 07:45:05.902982 1 server.go:555] Version: v1.14.1
I0817 07:45:05.922512 1 conntrack.go:52] Setting nf_conntrack_max to 131072
I0817 07:45:05.923301 1 config.go:102] Starting endpoints config controller
I0817 07:45:05.923338 1 controller_utils.go:1027] Waiting for caches to sync for endpoints config controller
I0817 07:45:05.923354 1 config.go:202] Starting service config controller
I0817 07:45:05.923416 1 controller_utils.go:1027] Waiting for caches to sync for service config controller
I0817 07:45:06.023978 1 controller_utils.go:1034] Caches are synced for endpoints config controller
I0817 07:45:06.024736 1 controller_utils.go:1034] Caches are synced for service config controller
（3）、service类型
- Service资源清单文件：
kind: Service # 资源类型
apiVersion: v1 # 资源版本
metadata: # 元数据
name: service # 资源名称
namespace: dev # 命名空间
spec: # 描述
selector: # 标签选择器，用于确定当前service代理哪些pod
app: nginx
type: # Service类型，指定service的访问方式
clusterIP: # 虚拟服务的ip地址
sessionAffinity: # session亲和性，支持ClientIP、None两个选项
ports: # 端口信息
- protocol: TCP
port: 3017 # service端口
targetPort: 5003 # pod端口
nodePort: 31122 # 主机端口
**Cluster IP:**默认值,它是Kubernetes系统自动分配的虚拟Ip,只能在集群内部访问
**NodePort:**将Service通过指定的Node上的端口暴露给外部,通过此方法,就可以在集群外部访问服务
**LoadBalancer:**使用外接负载均衡器完成到服务的负载分发,注意此模式需要外部云环境支持
**ExternalName:**把集群外部的服务引入集群内部,直接使用
（4）、service的使用
实验环境准备:
在使用service之前,首先利用Deployment创建出3个Pod,注意要为pod设置,
app=nginx-pod的标签
- 创建deployment.yaml文件:
cat <<EOF > deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: pc-deployment
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
EOF
#创建pod
[root@xnode1 ~]# kubectl create -f deployment.yaml
deployment.apps/pc-deployment created
#查看
[root@xnode1 ~]# kubectl get pod -n dev -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pc-deployment-d46fb9b9-5znzj 1/1 Running 0 51s 10.244.2.10 xnode3 <none> <none>
pc-deployment-d46fb9b9-bl89n 1/1 Running 0 51s 10.244.1.76 xnode2 <none> <none>
pc-deployment-d46fb9b9-q5n7x 1/1 Running 0 51s 10.244.1.77 xnode2 <none> <none>
#进入pod,修改nginx的访问页面
[root@xnode1 ~]# kubectl exec -it pc-deployment-d46fb9b9-5znzj -n dev /bin/sh
# echo "10.244.2.10" > /usr/share/nginx/html/index.html
# exit
[root@xnode1 ~]# kubectl exec -it pc-deployment-d46fb9b9-bl89n -n dev /bin/sh
# echo "10.244.1.76" > /usr/share/nginx/html/index.html
# exit
[root@xnode1 ~]# kubectl exec -it pc-deployment-d46fb9b9-q5n7x -n dev /bin/sh
# echo "10.244.1.77" > /usr/share/nginx/html/index.html
# exit
#访问测试
[root@xnode1 ~]# curl 10.244.2.10:80
10.244.2.10
[root@xnode1 ~]# curl 10.244.1.76:80
10.244.1.76
[root@xnode1 ~]# curl 10.244.1.77:80
10.244.1.77
（5）、ClusterIP类型的Services
- 创建service-cluster.yaml文件
cat <<EOF > service-cluster.yaml
apiVersion: v1
kind: Service
metadata:
name: service-clusterip
namespace: dev
spec:
selector:
app: nginx-pod
clusterIP: 10.97.97.97 # service的ip地址，如果不写，默认会生成一个
type: ClusterIP
ports:
- port: 80 # Service端口
targetPort: 80 # pod端口
EOF
- 创建ClusterIP:
#创建ClusterIP
[root@xnode1 ~]# kubectl create -f service-cluster.yaml
service/service-clusterip created
#查看
[root@xnode1 ~]# kubectl get service -n dev -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
nginx NodePort 10.106.191.78 <none> 80:32647/TCP 2d5h run=nginx
service-clusterip ClusterIP 10.97.97.97 <none> 80/TCP 56s app=nginx-pod
#查看clusterIP的详细信息
[root@xnode1 ~]# kubectl describe service -n dev service-clusterip
Name: service-clusterip
Namespace: dev
Labels: <none>
Annotations: <none>
Selector: app=nginx-pod
Type: ClusterIP
IP: 10.97.97.97
Port: <unset> 80/TCP
TargetPort: 80/TCP
Endpoints: 10.244.1.78:80,10.244.1.79:80,10.244.2.11:80
Session Affinity: None
Events: <none>
#查看ipvs的映射规则
[root@xnode1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.97.97.97:80 rr
-> 10.244.1.78:80 Masq 1 0 0
-> 10.244.1.79:80 Masq 1 0 0
-> 10.244.2.11:80 Masq 1 0 0
#访问测试
[root@xnode1 ~]# curl 10.97.97.97:80
10.244.1.79
- Endpoint:
Endpoint :
①.Endpoint是kubernetes中的一个资源对象,存储在etcd中,存储在ectd中,用来记录一个 service对应的所有pod的访问地址,它是根据 service配置文件中的selector描述产生的.
②.一个 service 由一组Pod组成,这些Pod通过 Endpoints暴露出来, Endpoint是实现实际服务的端点集合.换句话说,service和pod之间的联系是通过 Endpoint实现的
- 负载分发策略:
对service的访问被分发到了后端的Pod上去,目前kubernetes提供了两种负载分发策略
- 如果不定义,默认使用 kube-proxy的策略,比如说 轮询、随机
- 基于客户端地址的回话保持模式，即来自同一个客户端发起的所有请求都会转发到固定的一个Pod上，此模式可以使在spec中添加
sessionAffinity:ClientIP选项
#修改service-cluster.yaml文件
apiVersion: v1
kind: Service
metadata:
name: service-clusterip ==>添加此行
namespace: dev
spec:
sessionAffinity: ClientIP
selector:
app: nginx-pod
clusterIP: 10.97.97.97 # service的ip地址，如果不写，默认会生成一个
type: ClusterIP
ports:
- port: 80 # Service端口
targetPort: 80 # pod端口
#删除之前创建的clusterip service服务
[root@xnode1 ~]# kubectl delete -f service-cluster.yaml
service "service-clusterip" deleted
#启动新的服务
[root@xnode1 ~]# kubectl create -f service-cluster.yaml
service/service-clusterip created
#查看ipvs的映射规则: 【persistent 代表持久】
[root@xnode1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.97.97.97:80 rr persistent 10800
-> 10.244.1.78:80 Masq 1 0 0
-> 10.244.1.79:80 Masq 1 0 0
-> 10.244.2.11:80 Masq 1 0 0
#循环访问测试:发现这个时候的访问都是固定的一个Pod
[root@xnode1 ~]# while true;do curl 10.97.97.97; sleep 5; done;
10.244.2.11
10.244.2.11
10.244.2.11
10.244.2.11
10.244.2.11
10.244.2.11
#删除service
[root@xnode1 ~]# kubectl delete -f service-cluster.yaml
service "service-clusterip" deleted
（6）、HeadLiness类型的Service
HeadLiness Service:
在某些场景中，开发人员可能不想使用Service提供的负载均衡功能，而希望自己来控制负载均衡策略，针对这种情况，kubernetes提供了HeadLiness Service,这类Service不会分配ClusterIP,如果想要访问service,只能通过service的域名进行查询.
- 创建service-headliness.yaml
cat <<EOF > service-headliness.yaml
apiVersion: v1
kind: Service
metadata:
name: service-headliness
namespace: dev
spec:
selector:
app: nginx-pod
clusterIP: None # 将clusterIP设置为None，即可创建headliness Service
type: ClusterIP
ports:
- port: 80
targetPort: 80
EOF
- 创建服务并查看
#创建service
[root@xnode1 ~]# kubectl create -f service-headliness.yaml
service/service-headliness created
#获取service,发现CLUSTER-IP未分配
[root@xnode1 ~]# kubectl get svc -n dev service-headliness -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service-headliness ClusterIP None <none> 80/TCP 93s app=nginx-pod
#查看service:service-headliness的详细信息
[root@xnode1 ~]# kubectl describe svc -n dev service-headliness
Name: service-headliness
Namespace: dev
Labels: <none>
Annotations: <none>
Selector: app=nginx-pod
Type: ClusterIP
IP: None
Port: <unset> 80/TCP
TargetPort: 80/TCP
Endpoints: 10.244.1.78:80,10.244.1.79:80,10.244.2.11:80
Session Affinity: None
Events: <none>
#查看pod[root@xnode1 ~]# kubectl get pod -n dev
NAME READY STATUS RESTARTS AGE
pc-deployment-d46fb9b9-5znzj 1/1 Running 1 8h
pc-deployment-d46fb9b9-bl89n 1/1 Running 1 8h
pc-deployment-d46fb9b9-q5n7x 1/1 Running 1 8h
#进入一个pod查看域名解析情况
[root@xnode1 ~]# kubectl exec -it -n dev pc-deployment-d46fb9b9-5znzj /bin/sh
# grep ^search /etc/resolv.conf
search dev.svc.cluster.local svc.cluster.local cluster.local
# grep ^nameserver /etc/resolv.conf
nameserver 10.96.0.10
#dig 'pod:cat /etc/resolv.conf ^nameserver'、'service名字'、'命名空间'、'pod:cat /etc/resolv.conf ^search'
[root@xnode1 ~]# dig @10.96.0.10 serivce-headliness.dev.svc.cluster.local
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7 <<>> @10.96.0.10 serivce-headliness.dev.svc.cluster.local
; (1 server found)
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 62821
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;serivce-headliness.dev.svc.cluster.local. IN A
;; AUTHORITY SECTION:
cluster.local. 5 IN SOA ns.dns.cluster.local. hostmaster.cluster.local. 1661424260 7200 1800 86400 5
;; Query time: 16 msec
;; SERVER: 10.96.0.10#53(10.96.0.10)
;; WHEN: Thu Aug 25 06:52:36 EDT 2022
;; MSG SIZE rcvd: 162
（7）、NodePort类型的Service
NodePort:
在之前的样例中,创建的service的IP地址只有集群内部才可以访问,如果希望service暴露给集群外部使用,那么就要使用到另外一种类型的service,称为NodePort类型.NodePort的工作原理其实就是 将service的端口映射到Node的一个端口上,然后就可以通过
NodeIP:NodePort来访问Service了.
- 创建一个service-nodeport.yaml文件
cat <<EOF > service-nodeport.yaml
apiVersion: v1
kind: Service
metadata:
name: service-nodeport
namespace: dev
spec:
selector:
app: nginx-pod
type: NodePort # service类型
ports:
- port: 80
nodePort: 30002 # 指定绑定的node的端口(默认的取值范围是：30000-32767), 如果不指定，会默认分配
targetPort: 80
EOF
- 创建service:nodeport
#创建NodePort
[root@xnode1 ~]# kubectl create -f service-nodeport.yaml
service/service-nodeport created
#查看service服务
[root@xnode1 ~]# kubectl get svc -n dev -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service-headliness ClusterIP None <none> 80/TCP 38m app=nginx-pod
service-nodeport NodePort 10.97.55.214 <none> 80:30002/TCP 113s app=nginx-pod
#通过电脑浏览器访问集群中的nodeip的30002端口访问pod
[root@xnode1 ~]# curl 10.0.0.30:30002
10.244.2.11
（8）、LoadBalancer类型的Service
LoadBalancer:
LoadBalancer 和NodePort很相似,目的都是向外部暴露一个端口,区别在于 LoadBalancer会在集群的外部再来做一个负载均衡设备,而这个设备需要外部环境支持的,外部服务发送到这个设备上的请求,会被设备负载之后转发到集群中.
（9）、ExternalName类型的Service
ExternalName:
ExternalName类型的 Service用于引入集群外部的服务,它通过 externalName属性指定外部一个服务的地址,然后再集群内部访问此service就可以访问到外部的服务了.
- 创建一个service-externalname.yaml
cat <<EOF > service-external.yaml
apiVersion: v1
kind: Service
metadata:
name: service-externalname
namespace: dev
spec:
type: ExternalName # service类型
externalName: www.baidu.com #改成ip地址也可以
EOF
- 创建一个service:
[root@xnode1 ~]# kubectl create -f service-external.yaml
service/service-externalname created
#查看svc
[root@xnode1 ~]# kubectl get svc -n dev -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service-externalname ExternalName <none> www.baidu.com <none> 35s <none>
#查看详细信息
[root@xnode1 ~]# kubectl describe svc -n dev service-externalname
Name: service-externalname
Namespace: dev
Labels: <none>
Annotations: <none>
Selector: <none>
Type: ExternalName
IP:
External Name: www.baidu.com
Session Affinity: None
Events: <none>
#解析域名
[root@xnode1 ~]# dig @10.96.0.10 service-externalname.dev.svc.cluster.local
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7 <<>> @10.96.0.10 service-externalname.dev.svc.cluster.local
; (1 server found)
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 50647
;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;service-externalname.dev.svc.cluster.local. IN A
;; ANSWER SECTION:
service-externalname.dev.svc.cluster.local. 5 IN CNAME www.baidu.com.
www.baidu.com. 5 IN CNAME www.a.shifen.com.
www.a.shifen.com. 5 IN A 220.181.38.149
www.a.shifen.com. 5 IN A 220.181.38.150
;; Query time: 56 msec
;; SERVER: 10.96.0.10#53(10.96.0.10)
;; WHEN: Thu Aug 25 22:19:37 EDT 2022
;; MSG SIZE rcvd: 247
（10）、Ingress介绍
在前面的提到了,service对集群之外暴露服务的主要两种方式: NodePort 、LoadBalancer，但是这两种方式都有一定的缺点：
- NodePort方式的缺点是会占用很多的集群机器的端口,那么当集群服务变多的时候,这个缺点就会愈发明显.
- LB 的方式的缺点是每个service需要一个LB,浪费,麻烦,并且需要kubernetes之外设备的支持
基于这种现状kubernetes提供了Ingress资源对象,Ingress只需要一个 NodePort或者一个 LB就可以满足暴露多个service的需求
工作流程:
Ingress:
Ingress相当于一个7层负载均衡器,是kubernetes对反向代理的一个抽象,它的工作原理类似于Nginx,可以理解成在 Ingress里面建立诸多的映射规则,Ingress Controller通过监听这些配置规则并转换成Nginx的反向代理配置,然后对外提供服务.
- **Ingress:**kubernetes中的一个对象,作用是定义四请求如何转发到service的规则
- **Ingress controller:**具体实现反向代理以及负载均衡的程序,对Ingress定义的规则进行解析,根据配置的规则来实现请求转发,实现的方式很多,比如Nginx,Contour,Haproxy… …
Ingress(以Nginx为例)的工作原理:
- 用户编写Ingress规则,说明哪个域名对应的kubernetes集群中的哪个service
- Ingress controller动态感知 Ingress服务规则的变化,然后生成一段对应的Nginx反向代理配置
- Ingress controller将会生成的Nginx配置写入到一个运行着的Nginx服务中,并动态更新
- 到了这里,真正意义上工作的就是一个Nginx了,内部配置了用户定义请求转发规则
（11）、Ingress的使用
- 1.环境准备：
[root@xnode1 ~]# mkdir ingress-controller
[root@xnode1 ~]# wget -P ingress-controller/ https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml
[root@xnode1 ~]# wget -P ingress-controller/ https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/provider/baremetal/service-nodeport.yaml
# 创建ingress-nginx
[root@xnode1 ingress-controller]# kubectl apply -f ./
namespace/ingress-nginx created
configmap/nginx-configuration created
configmap/tcp-services created
configmap/udp-services created
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
deployment.apps/nginx-ingress-controller created
limitrange/ingress-nginx created
service/ingress-nginx created
# 查看pod
[root@xnode1 ingress-controller]# kubectl get pod -n ingress-nginx
NAME READY STATUS RESTARTS AGE
nginx-ingress-controller-767bbbd54f-rn5n2 1/1 Running 0 97s
# 查看service
[root@xnode1 ingress-controller]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx NodePort 10.110.182.175 <none> 80:32599/TCP,443:32410/TCP 26s
- 2.准备service和pod
#创建tomcat-nginx. yaml
cat <<EOF > tomcat-nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: tomcat-deployment
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
app: tomcat-pod
template:
metadata:
labels:
app: tomcat-pod
spec:
containers:
- name: tomcat
image: tomcat:8.5-jre10-slim
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: nginx-service
namespace: dev
spec:
selector:
app: nginx-pod
clusterIP: None
type: ClusterIP
ports:
- port: 80
targetPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: tomcat-service
namespace: dev
spec:
selector:
app: tomcat-pod
clusterIP: None
type: ClusterIP
ports:
- port: 8080
targetPort: 8080
EOF
#创建
[root@xnode1 ~]# kubectl create -f tomcat-nginx.yaml
deployment.apps/nginx-deployment created
deployment.apps/tomcat-deployment created
service/nginx-service created
service/tomcat-service created
#查看
[root@xnode1 ~]# kubectl get svc -n dev
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-service ClusterIP None <none> 80/TCP 29s
tomcat-service ClusterIP None <none> 8080/TCP 29s
- 3.http的创建
#创建ingress-http.yaml
cat <<EOF > ingress-http.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-http
namespace: dev
spec:
rules:
- host: nginx.itheima.com
http:
paths:
- path: /
backend:
serviceName: nginx-service
servicePort: 80
- host: tomcat.itheima.com
http:
paths:
- path: /
backend:
serviceName: tomcat-service
servicePort: 8080
EOF
#创建
[root@xnode1 ~]# kubectl create -f ingress-http.yaml
ingress.extensions/ingress-http created
#查看
[root@xnode1 ~]# kubectl get ingresses ingress-http -n dev
NAME HOSTS ADDRESS PORTS AGE
ingress-http nginx.itheima.com,tomcat.itheima.com 80 25s
#查看http服务的详情
[root@xnode1 ~]# kubectl describe ingresses ingress-http -n dev
... ...
Rules:
Host Path Backends
---- ---- --------
nginx.itheima.com
/ nginx-service:80 (10.244.1.91:80,10.244.1.92:80,10.244.2.14:80)
tomcat.itheima.com
/ tomcat-service:8080 (10.244.1.89:8080,10.244.1.90:8080,10.244.2.15:8080)
... ...
#修改Windows目录:C:\Windows\System32\drivers\etc下的hosts文件,添加域名映射
10.0.0.30 nginx.itheima.com
10.0.0.30 tomcat.itheima.com
使用主机访问http://nginx.itheima.com:31546/
访问 http://tomcat.itheima.com:31546/
-
4.https代理:
-
创建证书和秘钥
[root@xnode1 ~]# openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/C=CN/ST=BJ/L=BJ/O=nginx/CN=itheima.com"
Generating a 2048 bit RSA private key
............+++
..................+++
writing new private key to 'tls.key'
-----
[root@xnode1 ~]# kubectl create secret tls tls-secret --key tls.key --cert tls.crt
secret/tls-secret created
- 创建ingress-https.yaml
cat <<EOF > ingress-https.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-https
namespace: dev
spec:
tls:
- hosts:
- nginx.itheima.com
- tomcat.itheima.com
secretName: tls-secret # 指定秘钥
rules:
- host: nginx.itheima.com
http:
paths:
- path: /
backend:
serviceName: nginx-service
servicePort: 80
- host: tomcat.itheima.com
http:
paths:
- path: /
backend:
serviceName: tomcat-service
servicePort: 8080
EOF
#创建
[root@xnode1 ~]# kubectl create -f ingress-https.yaml
ingress.extensions/ingress-https created
#查看
[root@xnode1 ~]# kubectl get ingresses ingress-https -n dev
NAME HOSTS ADDRESS PORTS AGE
ingress-https nginx.itheima.com,tomcat.itheima.com 10.102.125.70 80, 443 13s
#查看https的详细信息
[root@xnode1 ~]# kubectl describe ingresses ingress-https -n dev
...
TLS:
tls-secret terminates nginx.itheima.com,tomcat.itheima.com
Rules:
Host Path Backends
---- ---- --------
nginx.itheima.com
/ nginx-service:80 (10.244.1.91:80,10.244.1.92:80,10.244.2.14:80)
tomcat.itheima.com
/ tomcat-service:8080 (10.244.1.89:8080,10.244.1.90:8080,10.244.2.15:8080)
...
#通过浏览器访问 https://tomcat.itheima.com:31034/ https://nginx.itheima.com:31034/
十三、数据存储
在前面已经提到，容器的生命周期可能很短，会被频繁地创建和销毁。那么容器在销毁时，保存在容器中的数据也会被清除。这种结果对用户来说，在某些情况下是不乐意看到的。为了持久化保存容器的数据，kubernetes引入了Volume的概念。
Volume:
Volume是Pod中能够被多个容器访问的共享目录,它被定义在pod上,然后被一个Pod里的多个容器挂载到具体的文件目录下,kubernetes通过 Volume实现同一个pod中不同容器之间的数据共享以及数据化持久化存储, Volume的生命容器不与Pod中单个容器的生命周期相关,当容器终止或者重启时, Volume中的数据也不会丢失.
kubernetes的 Volume支持多种类型,常见的有下面几个:
- 简单存储：EmptyDir、HostPath、NFS
- 高级存储：PV、PVC
- 配置存储：ConfigMap、Secret
（1）、EmptyDir
EmptyDir:
EmptyDir是在pod被分配到Node时创建的,它的初始内容为空,并且不误指定宿主机上对应的目录文件,因为kubernetes会自动分配一个目录,当pod销毁时, EmptyDir中的数据也会被永久删除, EmptyDir用途如下.
- 临时空间,例如用于某些应用程序运行时所需的临时目录,且无需永久保留,作为容器之间共享目录进行的数据存储可以使用 EmptyDir
- 一个容器需要从另一个容器中获取数据的目录(多容器共享目录)
在一个Pod中准备两个容器nginx和busybox，然后声明一个Volume分别挂在到两个容器的目录中，然后nginx容器负责向Volume中写日志，busybox中通过命令将日志内容读到控制台。
- 创建一个volume-emptydir.yaml
cat <<EOF > volume-emptydir.yaml
apiVersion: v1
kind: Pod
metadata:
name: volume-emptydir
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
volumeMounts: # 将logs-volume挂在到nginx容器中，对应的目录为 /var/log/nginx
- name: logs-volume
mountPath: /var/log/nginx
- name: busybox
image: busybox:1.30
command: ["/bin/sh","-c","tail -f /logs/access.log"] # 初始命令，动态读取指定文件中内容
volumeMounts: # 将logs-volume 挂在到busybox容器中，对应的目录为 /logs
- name: logs-volume
mountPath: /logs
volumes: # 声明volume， name为logs-volume，类型为emptyDir
- name: logs-volume
emptyDir: {}
EOF
#创建pod
[root@xnode1 ~]# kubectl create -f volume-emptydir.yaml
pod/volume-emptydir created
#查看
[root@xnode1 ~]# kubectl get pods -n dev volume-emptydir -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
volume-emptydir 2/2 Running 0 38s 10.244.2.18 xnode3 <none> <none>
#通过podip访问nginx
[root@xnode1 ~]# curl 10.244.2.18:80
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
#通过kubectl logs命令查看指定容器的标准输出
[root@xnode1 ~]# kubectl logs -f -n dev volume-emptydir -c busybox
10.244.0.0 - - [30/Aug/2022:16:30:33 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
（2）、HostPath
HostPath:
HostPath就是将Node主机中一个实际目录挂在Pod中,以供容器使用,这样的设计就可以保证Pod销毁但是数据依旧可以存在于Node主机上
和前面讲过的 EmptyDir不同的是, EmptyDir中的数据不会持久化,它会随着Pod的结束而销毁.
- 创建一个volume-hostpath.yaml文件
cat <<EOF > volume-hostpath.yaml
apiVersion: v1
kind: Pod
metadata:
name: volume-hostpath
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
volumeMounts:
- name: logs-volume
mountPath: /var/log/nginx
- name: busybox
image: busybox:1.30
command: ["/bin/sh","-c","tail -f /logs/access.log"]
volumeMounts:
- name: logs-volume
mountPath: /logs
volumes:
- name: logs-volume
hostPath:
path: /root/logs
type: DirectoryOrCreate # 目录存在就使用，不存在就先创建后使用
EOF
关于 Type的说明:
- DirectoryOrCreate:目录存在就使用,不存在就先创建后使用
- **Directory:**目录必须存在
- **FileOrCreate:**文件存在就使用,不存在就先创建后使用
- **File:**文件必须存在
- **Socket:**Unix套接字必须存在
- **CharDevice:**字符设备必须存在
- **BlockDevice:**块设备必须存在
#创建pod
[root@xnode1 ~]# kubectl create -f volume-hostpath.yaml
pod/volume-hostpath created
#查看pod
[root@xnode1 ~]# kubectl get pods -n dev volume-hostpath -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
volume-hostpath 2/2 Running 0 20s 10.244.1.101 xnode2 <none> <none>
#访问nginx
[root@xnode1 ~]# curl 10.244.1.101:80
... ...
#查看xnode2节点的日志文件
[root@xnode2 ~]# cat /root/logs/access.log
10.244.0.0 - - [30/Aug/2022:08:11:20 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
（3）、NFS
NFS:
NFS是一个网络文件存储系统,可以搭建一台NFS服务器,然后将Pod中的存储直接连接到NFS系统上,这样的话,无论Pod在节点上怎么转移,只要Node和NFS的对接没有问题,数据就可以成功访问.
HostPath可以解决数据持久化的问题,但是一旦Node节点故障,Pod如果转移到了其它节点,又会出现问题,此时需要准备单独的网络存储系统,比如常用的NFS和CIFS
a）、准备搭建NFS服务,这里选择在master主节点上部署nfs服务
#安装nfs服务
[root@xnode1 ~]# yum install nfs-utils.x86_64 -y
#创建一个共享目录
[root@xnode1 ~]# mkdir /root/data/nfs -pv
#将共享目录以读写权限暴露给10.0.0.0/24网段下的所有主机
[root@xnode1 ~]# echo "/root/data/nfs/ 10.0.0.0/24(rw,no_root_squash)" >> /etc/exports
[root@xnode1 ~]# tail -1 /etc/exports
/root/data/nfs/ 10.0.0.0/24(rw,no_root_squash)
#启动nfs服务
[root@xnode1 ~]# systemctl start nfs
[root@xnode1 ~]# systemctl enable nfs
b）、在xnode2、xnode3节点上安装nfs
#安装nfs服务,不要启动
yum install nfs-utils.x86_64 -y
c）、编写volume-nfs.yaml
cat <<EOF > volume-nfs.yaml
apiVersion: v1
kind: Pod
metadata:
name: volume-nfs
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
volumeMounts:
- name: logs-volume
mountPath: /var/log/nginx
- name: busybox
image: busybox:1.30
command: ["/bin/sh","-c","tail -f /logs/access.log"]
volumeMounts:
- name: logs-volume
mountPath: /logs
volumes:
- name: logs-volume
nfs:
server: 10.0.0.30 #nfs服务器地址
path: /root/data/nfs #共享文件路径
EOF
d）、运行Pod查看运行结果
#创建Pod
[root@xnode1 ~]# kubectl create -f volume-nfs.yaml
pod/volume-nfs created
#查看Pod
[root@xnode1 ~]# kubectl get pod -n dev -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
volume-nfs 2/2 Running 0 12s 10.244.1.102 xnode2 <none> <none>
#访问nginx服务
[root@xnode1 nfs]# curl 10.244.1.102:80
...
#查看nfs服务器共享目录及日志
[root@xnode1 ~]# cd /root/data/nfs/
[root@xnode1 nfs]# ls
access.log error.log
[root@xnode1 nfs]# cat access.log
10.244.0.0 - - [30/Aug/2022:08:53:57 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
（4）、高级存储
前面以及学习了使用NFS提供的存储,此时就要求用户会搭建NFS系统,并且会在yaml配置nfs,由于kubernetes支持的存储系统有很多,要求客户全都掌握,显然不现实,为了能够方便用户的使用,kubernetes推出了PV、PVC这两种资源对象。
PV：
**PV(Persistent Volume)**是持久卷的意思,是对底层共享存储的一种抽象,一般情况下 PV由kubernetes管理员进行创建和配置,它与底层具体的共享存储技术相关,并通过插件完成共享存储的对接
PVC(Persistent Volume Claim):是持久卷声明的意思,是用户对存储需求的一种声明,换句话说PVC其实就是用户向kubernetes系统发出的一种资源需求申请.
使用了 PV和 PVC之后,工作可以得到进一步的细分
- 存储: 存储工程师维护
- **PV:**kubernetes管理员负责维护
- **PVC:**kubernetes用户维护
（5）、PV
PV是存储资源的抽象,下面是资源清单文件
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv2
spec:
nfs: # 存储类型，与底层真正存储对应
capacity: # 存储能力，目前只支持存储空间的设置
storage: 2Gi
accessModes: # 访问模式
storageClassName: # 存储类别
persistentVolumeReclaimPolicy: # 回收策略
PV 的关键配置参数说明：
-
存储类型
底层实际存储的类型，kubernetes支持多种存储类型，每种存储类型的配置都有所差异
-
存储能力（capacity）
目前只支持存储空间的设置( storage=1Gi )，不过未来可能会加入IOPS、吞吐量等指标的配置
-
访问模式（accessModes）
用于描述用户应用对存储资源的访问权限，访问权限包括下面几种方式：
- ReadWriteOnce（RWO）：读写权限，但是只能被单个节点挂载
- ReadOnlyMany（ROX）： 只读权限，可以被多个节点挂载
- ReadWriteMany（RWX）：读写权限，可以被多个节点挂载
需要注意的是，底层不同的存储类型可能支持的访问模式不同 -
回收策略（persistentVolumeReclaimPolicy）
当PV不再被使用了之后，对其的处理方式。目前支持三种策略：
- Retain （保留） 保留数据，需要管理员手工清理数据
- Recycle（回收） 清除 PV 中的数据，效果相当于执行 rm -rf /thevolume/*
- Delete （删除） 与 PV 相连的后端存储完成 volume 的删除操作，当然这常见于云服务商的存储服务
需要注意的是，底层不同的存储类型可能支持的回收策略不同 -
存储类别
PV可以通过storageClassName参数指定一个存储类别
- 具有特定类别的PV只能与请求了该类别的PVC进行绑定
- 未设定类别的PV则只能与不请求任何类别的PVC进行绑定
-
状态（status）
一个 PV 的生命周期中，可能会处于4中不同的阶段：
- Available（可用）： 表示可用状态，还未被任何 PVC 绑定
- Bound（已绑定）： 表示 PV 已经被 PVC 绑定
- Released（已释放）： 表示 PVC 被删除，但是资源还未被集群重新声明
- Failed（失败）： 表示该 PV 的自动回收失败
-
创建一个PV
使用NFS作为存储,来演示PV的使用,创建3个PV,对应的NFS中的3个暴露的路径)
a）、准备NFS环境
#创建目录
[root@xnode1 ~]# mkdir /root/data/{pv1,pv2,pv3} -pv
#暴露服务
[root@xnode1 ~]# cat <<EOF >> /etc/exports
/root/data/pv1 10.0.0.0/24(rw,no_root_squash)
/root/data/pv2 10.0.0.0/24(rw,no_root_squash)
/root/data/pv3 10.0.0.0/24(rw,no_root_squash)
EOF
#重启服务
[root@xnode1 ~]# systemctl restart nfs
b）、创建pv.yaml
cat <<EOF > pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv1
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
nfs:
path: /root/data/pv1
server: 10.0.0.30
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv2
spec:
capacity:
storage: 2Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
nfs:
path: /root/data/pv2
server: 10.0.0.30
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv3
spec:
capacity:
storage: 3Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
nfs:
path: /root/data/pv3
server: 10.0.0.30
EOF
#创建pv
[root@xnode1 ~]# kubectl create -f pv.yaml
persistentvolume/pv1 created
persistentvolume/pv2 created
persistentvolume/pv3 created
#查看pv
[root@xnode1 ~]# kubectl get pv -o wide
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv1 1Gi RWX Retain Available 30s
pv2 2Gi RWX Retain Available 30s
pv3 3Gi RWX Retain Available 30s
（6）、PVC
PVC:
PVC是资源申请,用来声明对存储空间、访问模式、存储类别需求的信息，接下来我们查看一下清单文件.
#PVC清单文件
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc
namespace: dev
spec:
accessModes: # 访问模式
selector: # 采用标签对PV选择
storageClassName: # 存储类别
resources: # 请求空间
requests:
storage: 5Gi
PVC关键访问参数:
**访问模式(accessModes)😗*用于描述用户对应存储资源的访问权限
选择条件(selector):通过 Label Selector,可使PVC对于系统中已存在的PV进行筛选
存储类别(storageClassName): ** PVC在定义时可以设定需要的后端存储的类型,只有设置了该class的PV**才能被系统选出
**资源请求(Resources)😗*描述对资源存储的请求
a）、实验
- 1、创建pvc.yaml，申请pv
cat <<EOF > pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc1
namespace: dev
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc2
namespace: dev
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc3
namespace: dev
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
EOF
#创建pvc
[root@xnode1 ~]# kubectl create -f pvc.yaml
persistentvolumeclaim/pvc1 created
persistentvolumeclaim/pvc2 created
persistentvolumeclaim/pvc3 created
#查看pvc
[root@xnode1 ~]# kubectl get pvc -n dev -o wide
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
pvc1 Bound pv1 1Gi RWX 22s
pvc2 Bound pv2 2Gi RWX 22s
pvc3 Bound pv3 3Gi RWX 22s
#查看pv
[root@xnode1 ~]# kubectl get pv -n dev -o wide
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv1 1Gi RWX Retain Bound dev/pvc1 47h
pv2 2Gi RWX Retain Bound dev/pvc2 47h
pv3 3Gi RWX Retain Bound dev/pvc3 47h
- 2**、创建pods.yaml文件**
cat <<EOF > pods.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod1
namespace: dev
spec:
containers:
- name: busybox
image: busybox:1.30
command: ["/bin/sh","-c","while true;do echo pod1 >> /root/out.txt; sleep 10; done;"]
volumeMounts:
- name: volume
mountPath: /root/
volumes:
- name: volume
persistentVolumeClaim:
claimName: pvc1
readOnly: false
---
apiVersion: v1
kind: Pod
metadata:
name: pod2
namespace: dev
spec:
containers:
- name: busybox
image: busybox:1.30
command: ["/bin/sh","-c","while true;do echo pod2 >> /root/out.txt; sleep 10; done;"]
volumeMounts:
- name: volume
mountPath: /root/
volumes:
- name: volume
persistentVolumeClaim:
claimName: pvc2
readOnly: false
EOF
#创建pod
[root@xnode1 ~]# kubectl create -f pods.yaml
pod/pod1 created
pod/pod2 created
#查看pod
[root@xnode1 ~]# kubectl get pods -n dev -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod1 1/1 Running 0 54s 10.244.2.3 xnode3 <none> <none>
pod2 1/1 Running 0 54s 10.244.2.2 xnode3 <none> <none>
#查看pvc
[root@xnode1 ~]# kubectl get pvc -n dev -o wide
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
pvc1 Bound pv1 1Gi RWX 10m
pvc2 Bound pv2 2Gi RWX 10m
pvc3 Bound pv3 3Gi RWX 10m
#查看pv
[root@xnode1 ~]# kubectl get pv -n dev -o wide
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv1 1Gi RWX Retain Bound dev/pvc1 10m
pv2 2Gi RWX Retain Bound dev/pvc2 10m
pv3 3Gi RWX Retain Bound dev/pvc3 10m
#查看nfs之中的文件存储
[root@xnode1 ~]# more /root/data/pv1/out.txt
pod1
pod1
...
[root@xnode1 ~]# more /root/data/pv2/out.txt
pod2
pod2
...
（7）、生命周期
PV和PVC是对应的，PV和PVC之间的相互作用遵循以下的生命周期：
- 资源供应：管理员手动创建底层存储和PV
- 资源绑定：用户创建PVC，kubernetes负责根据PVC的声明去寻找PV，并绑定在用户定义好PVC之后，系统将根据PVC对存储资源的请求在已存在的PV中选择一个满足条件的
- 一旦找到,就将该PV与用户定义的PVC进行绑定,用户的应用就可以使用这个PVC了
- 如果找不到,PVC则会无限期处于Pending状态,直到等到系统管理员创建了一个符合其它要求的PV,PV一旦绑定到了某个PVC上,就会被这个PVC独占,不能再与其它的PVC进行绑定了.
-
资源使用:用户可在pod中像volume一样使用PVC
Pod使用Volume的定义,将PVC挂载到荣齐全内的某个路径进行使用.
-
资源释放:用户删除PVC来释放PV
当存储资源使用完毕后,用户可以删除PVC,与 该PVC绑定的PV将会被标记为"已释放",但还不能立刻与其它PVC进行绑定.通过之前PVC写入的数据还可能被留在存储设备上,只有在清除之后该PV才能再次使用.
-
资源回收：kubernetes根据pv设置的回收策略进行资源的回收
对于PV，管理员可以设定回收策略，用于设置与之绑定的PVC释放资源之后如何处理遗留数据的问题。只有PV的存储空间完成回收，才能供新的PVC绑定和使用
十四、配置存储
（1）、ConfigMap
ConfigMap:
ConfigMap是一种特殊的存储卷,它的作用主要是用来存储配置信息的.
创建 configmap.yaml,内容如下.
cat <<EOF > configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: configmap
namespace: dev
data:
info: |
username:admin
password:123456
EOF
- 使用此配置文件创建configmap
#创建configmap
[root@xnode1 ~]# kubectl create -f configmap.yaml
configmap/configmap created
#查看configmap的详情
[root@xnode1 ~]# kubectl describe configmaps -n dev configmap
Name: configmap
Namespace: dev
Labels: <none>
Annotations: <none>
Data
====
info:
----
username:admin
password:123456
Events: <none>
- 创建一个pod-configmap.yaml,将上面创建的configmap挂载进去
cat <<EOF > pod-configmap.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-configmap
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
volumeMounts: # 将configmap挂载到目录
- name: config
mountPath: /configmap/config
volumes: # 引用configmap
- name: config
configMap:
name: configmap
EOF
#创建pod
[root@xnode1 ~]# kubectl create -f pod-configmap.yaml
pod/pod-configmap created
#查看pod
[root@xnode1 ~]# kubectl get pods -n dev pod-configmap
NAME READY STATUS RESTARTS AGE
pod-configmap 1/1 Running 0 32s
#进入容器
[root@xnode1 ~]# kubectl exec -it -n dev pod-configmap /bin/sh
# cat /configmap/config/info
username:admin
password:123456
#可以看到映射已经成功,每个configmap都映射成了一个目录
#key --->文件 value--->文件中的内容
#如果我们这个时候更新configmap中的内容,容器中的文件内容也会随之更新
[root@xnode1 ~]# kubectl edit configmaps configmap -n dev
apiVersion: v1
data:
info: |
username:admin
password:admin --->修改
kind: ConfigMap
[root@xnode1 ~]# kubectl exec -it -n dev pod-configmap /bin/sh
# cat /configmap/config/info
username:admin
password:admin
（2）、Secret
kubernetes中还有一种和configmap很相似的配置存储工具,成为 Secret.它主要用来存储敏感信息,例如密码、秘钥、证书等。
- 首先对base64数据进行编码
[root@xnode1 ~]# echo -n 'admin' | base64
YWRtaW4=
[root@xnode1 ~]# echo -n '123456' | base64
MTIzNDU2
- 编写secret.yaml文件.并创建secret
cat <<EOF > secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: secret
namespace: dev
type: Opaque
data:
username: YWRtaW4=
password: MTIzNDU2
EOF
#创建secret
[root@xnode1 ~]# kubectl create -f secret.yaml
secret/secret created
#查看secret详情
[root@xnode1 ~]# kubectl describe secrets -n dev secret
Name: secret
Namespace: dev
Labels: <none>
Annotations: <none>
Type: Opaque
Data
====
password: 6 bytes
username: 5 bytes -->可以看到这里的用户名和密码以及经过加密了
- 创建一个pod-secret.yaml,将上面创建的secret挂载进去
cat <<EOF > pod-secret.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-secret
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
volumeMounts: # 将secret挂载到目录
- name: config
mountPath: /secret/config
volumes:
- name: config
secret:
secretName: secret
EOF
#创建pod
[root@xnode1 ~]# kubectl create -f pod-secret.yaml
pod/pod-secret created
#查看pod
[root@xnode1 ~]# kubectl get pod -n dev pod-secret -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod-secret 1/1 Running 0 29s 10.244.2.5 xnode3 <none> <none>
#进入容器.查看secret信息,发现已经自动解码了
[root@xnode1 ~]# kubectl exec -it -n dev pod-secret /bin/sh
# ls /secret/config/
password username
# cat /secret/config/username
admin
# cat /secret/config/password
123456
#至此已经利用secret实现了信息的编码
十五、安全认证
（1）、访问控制概述
为了保证kubernetes内的集群和pod的安全性,kubernetes推出了访问安全认证对 客户端进行 认证和鉴权操作.
客户端:
- **User Account:**一般是独立于kubernetes之外的其它服务管理的用户账号.
- **Services Account:**kubernetes管理的账号,用于Pod中的服务进程在访问kubernetes时提供身份标识.
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-dclTYIsv-1662121676524)(/image-20200520102949189.png)]
- 认证授权与准入控制:
APIServer是访问及管理资源对象的唯一入口.任何一个请求访问 APIServer,都要经过下面三个流程:
- **Authentication(认证)😗*身份鉴别,只有正确的账号才通过认证
- **Authorization(授权)😗*判断用户是否有访问权限对访问的资源执行特定的动作
- **Admission Control(准入控制)😗*用于补充授权机制以及实现更加精准的访问控制功能
（2）、认证管理
kubernetes提供了三种客户端认证方式:
**HTTP Base认证：**通过用户名+密码的方式认证
- 这种认证方式是把“用户名:密码”用BASE64算法进行编码后的字符串放在HTTP请求中的Header Authorization域里发送给服务端。服务端收到后进行解码，获取用户名及密码，然后进行用户身份认证的过程。
**HTTP Token认证：**通过一个Token来识别合法用户
- 这种认证方式是用一个很长的难以被模仿的字符串–Token来表明客户身份的一种方式。每个Token对应一个用户名，当客户端发起API调用请求时，需要在HTTP Header里放入Token，API Server接到Token后会跟服务器中保存的token进行比对，然后进行用户身份认证的过程。
**HTTPS证书认证：**基于CA根证书签名的双向数字证书认证方式
- HTTPS证书认证：基于CA根证书签名的双向数字证书认证方式
HTTPS认证大体分为3个过程:
- **1.证书申请和下发:**HTTPS通信双方的服务器向CA机构申请证书，CA机构下发根证书、服务端证书及私钥给申请者
- 2.客户端和服务端的双向认证:
a)、客户端向服务器端发起请求，服务端下发自己的证书给客户端，
客户端接收到证书后，通过私钥解密证书，在证书中获得服务端的公钥，
客户端利用服务器端的公钥认证证书中的信息，如果一致，则认可这个服务器
b)、 客户端发送自己的证书给服务器端，服务端接收到证书后，通过私钥解密证书，
在证书中获得客户端的公钥，并用该公钥认证证书信息，确认客户端是否合法
- 3.服务器端和客户端进行通信:
服务器端和客户端协商好加密方案后，客户端会产生一个随机的秘钥并加密，然后发送到服务器端。服务器端接收这个秘钥后，双方接下来通信的所有内容都通过该随机秘钥加密
# 注意:
Kubernetes允许同时配置多种认证方式，只要其中任意一个方式认证通过即可
（3）、授权管理
授权在认证成功之后,通过认证就可以知道用户是谁,然后kubernetes会根据事先定义的授权策略来决定用户是否有权限访问,这个过程就称之为授权.
每个发送到ApiServer的请求都带上了用户和资源的信息：比如发送请求的用户、请求的路径、请求的动作等，授权就是根据这些信息和授权策略进行比较，如果符合策略，则认为授权通过，否则会返回错误。
API Server目前支持以下几种授权策略：
- AlwaysDeny：表示拒绝所有请求，一般用于测试
- AlwaysAllow：允许接收所有请求，相当于集群不需要授权流程（Kubernetes默认的策略）
- ABAC：基于属性的访问控制，表示使用用户配置的授权规则对用户请求进行匹配和控制
- Webhook：通过调用外部REST服务对用户进行授权
- Node：是一种专用模式，用于对kubelet发出的请求进行访问控制
- RBAC：基于角色的访问控制（kubeadm安装方式下的默认选项）
RBAC(Role-Based Access Control) 基于角色的访问控制，主要是在描述一件事情：给哪些对象授予了哪些权限
其中涉及到了下面几个概念：
- 对象：User、Groups、ServiceAccount
- 角色：代表着一组定义在资源上的可操作动作(权限)的集合
- 绑定：将定义好的角色跟用户绑定在一起
RBAC引入了4个顶级资源对象:
- **Role、ClusterRole：**角色，用于指定一组权限
- **RoleBinding、ClusterRoleBinding：**角色绑定，用于将角色（权限）赋予给对象
**Role、ClusterRole：**一个角色就是一组权限的集合，这里的权限都是许可形式的（白名单）
# Role只能对命名空间内的资源进行授权，需要指定nameapce
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
namespace: dev
name: authorization-role
rules:
- apiGroups: [""] # 支持的API组列表,"" 空字符串，表示核心API群
resources: ["pods"] # 支持的资源对象列表
verbs: ["get", "watch", "list"] # 允许的对资源对象的操作方法列表
# ClusterRole可以对集群范围内资源、跨namespaces的范围资源、非资源类型进行授权
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: authorization-clusterrole
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list"]
需要详细说明的是，rules中的参数：
-
apiGroups: 支持的API组列表
"","apps", "autoscaling", "batch" -
resources：支持的资源对象列表
"services", "endpoints", "pods","secrets","configmaps","crontabs","deployments","jobs", "nodes","rolebindings","clusterroles","daemonsets","replicasets","statefulsets", "horizontalpodautoscalers","replicationcontrollers","cronjobs" -
verbs：对资源对象的操作方法列表
"get", "list", "watch", "create", "update", "patch", "delete", "exec"
RoleBinding、ClusterRoleBinding
角色绑定用来把一个角色绑定到一个目标对象上，绑定目标可以是User、Group或者ServiceAccount。
# RoleBinding可以将同一namespace中的subject绑定到某个Role下，则此subject即具有该Role定义的权限
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: authorization-role-binding
namespace: dev
subjects:
- kind: User
name: heima
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: authorization-role
apiGroup: rbac.authorization.k8s.io
# ClusterRoleBinding在整个集群级别和所有namespaces将特定的subject与ClusterRole绑定，授予权限
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: authorization-clusterrole-binding
subjects:
- kind: User
name: heima
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: authorization-clusterrole
apiGroup: rbac.authorization.k8s.io
RoleBinding引用ClusterRole进行授权
RoleBinding可以引用 ClusterRole,对属于同一命名空间内 ClusterRole定义的资源主体进行授权.
一种很常用的做法就是，集群管理员为集群范围预定义好一组角色（ClusterRole），然后在多个命名空间中重复使用这些ClusterRole。这样可以大幅提高授权管理工作效率，也使得各个命名空间下的基础性授权规则与使用体验保持一致。
# 虽然authorization-clusterrole是一个集群角色，但是因为使用了RoleBinding
# 所以heima只能读取dev命名空间中的资源
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: authorization-role-binding-ns
namespace: dev
subjects:
- kind: User
name: heima
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: authorization-clusterrole
apiGroup: rbac.authorization.k8s.io
实战:创建一个只能管理dev空间下的资源的账号
- 1、创建账号
# 1)、创建证书
[root@xnode1 ~]# mkdir -p /etc/kubernetes/pki
[root@xnode1 ~]# cd /etc/kubernetes/pki/
[root@xnode1 pki]# (umask 077;openssl genrsa -out devman.key 2048)
Generating RSA private key, 2048 bit long modulus
.......................................+++
....................................................................................................+++
e is 65537 (0x10001)
# 2）、用apiserver的证书签署
# 签名申请，申请的用户是devman，组是devgroup
[root@xnode1 pki]# openssl req -new -key devman.key -out devman.csr -subj "/CN=devman/O=devgroup"
#签署证书
[root@xnode1 pki]# openssl x509 -req -in devman.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out devman.crt -days 3650
Signature ok
subject=/CN=devman/O=devgroup
Getting CA Private Key
# 3）、设置集群、用户、上下文信息
[root@xnode1 pki]# kubectl config set-cluster kubernetes --embed-certs=true --certificate-authority=/etc/kubernetes/pki/ca.crt --server=https://10.0.0.30:6443
Cluster "kubernetes" set.
[root@xnode1 pki]# kubectl config set-credentials devman --embed-certs=true --client-certificate=/etc/kubernetes/pki/devman.crt --client-key=/etc/kubernetes/pki/devman.key
User "devman" set.
[root@xnode1 pki]# kubectl config set-context devman@kubernetes --cluster=kubernetes --user=devman
Context "devman@kubernetes" created.
# 将账户切换到devman
[root@xnode1 pki]# kubectl config use-context devman@kubernetes
Switched to context "devman@kubernetes".
#查看dev下的pod，发现无相关的权限访问
[root@xnode1 pki]# kubectl get pods -n dev
Error from server (Forbidden): pods is forbidden: User "devman" cannot list resource "pods" in API group "" in the namespace "dev"
#切换回admin账户
[root@xnode1 pki]# kubectl config use-context kubernetes-admin@kubernetes
Switched to context "kubernetes-admin@kubernetes".
- 2、创建Role和RoleBinding，为 devman用户授权
cat <<EOF > dev-role.yaml
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
namespace: dev
name: dev-role
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: authorization-role-binding
namespace: dev
subjects:
- kind: User
name: devman
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: dev-role
apiGroup: rbac.authorization.k8s.io
EOF
# 执行配置文件,赋予devman用户相应的角色和适当的权限
[root@xnode1 ~]# kubectl create -f dev-role.yaml
role.rbac.authorization.k8s.io/dev-role created
rolebinding.rbac.authorization.k8s.io/authorization-role-binding created
# 切换账户,再次验证
[root@xnode1 ~]# kubectl config use-context devman@kubernetes
Switched to context "devman@kubernetes".
[root@xnode1 ~]# kubectl get pods -n dev -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod-configmap 1/1 Running 0 159m 10.244.2.4 xnode3 <none> <none>
pod-secret 1/1 Running 0 135m 10.244.2.5 xnode3 <none> <none>
# 切换回admin账户
[root@xnode1 ~]# kubectl config use-context kubernetes-admin@kubernetes
Switched to context "kubernetes-admin@kubernetes".
（4）、准入控制
通过了前面的认证和授权之后，还需要经过准入控制处理通过之后，apiserver才会处理这个请求。
准入控制是一个可配置的控制器列表，可以通过在Api-Server上通过命令行设置选择执行哪些准入控制器：
--admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,
DefaultStorageClass,ResourceQuota,DefaultTolerationSeconds
只有当所有的准入控制器都检查通过之后，apiserver才执行该请求，否则返回拒绝。
当前可配置的Admission Control准入控制如下：
- AlwaysAdmit：允许所有请求
- AlwaysDeny：禁止所有请求，一般用于测试
- AlwaysPullImages：在启动容器之前总去下载镜像
- DenyExecOnPrivileged：它会拦截所有想在Privileged Container上执行命令的请求
- ImagePolicyWebhook：这个插件将允许后端的一个Webhook程序来完成admission controller的功能。
- Service Account：实现ServiceAccount实现了自动化
- SecurityContextDeny：这个插件将使用SecurityContext的Pod中的定义全部失效
- ResourceQuota：用于资源配额管理目的，观察所有请求，确保在namespace上的配额不会超标
- LimitRanger：用于资源限制管理，作用于namespace上，确保对Pod进行资源限制
- InitialResources：为未设置资源请求与限制的Pod，根据其镜像的历史资源的使用情况进行设置
- NamespaceLifecycle：如果尝试在一个不存在的namespace中创建资源对象，则该创建请求将被拒绝。当删除一个namespace时，系统将会删除该namespace中所有对象。
- DefaultStorageClass：为了实现共享存储的动态供应，为未指定StorageClass或PV的PVC尝试匹配默认的StorageClass，尽可能减少用户在申请PVC时所需了解的后端存储细节
- DefaultTolerationSeconds：这个插件为那些没有设置forgiveness tolerations并具有notready:NoExecute和unreachable:NoExecute两种taints的Pod设置默认的“容忍”时间，为5min
- PodSecurityPolicy：这个插件用于在创建或修改Pod时决定是否根据Pod的security context和可用的PodSecurityPolicy对Pod的安全策略进行控制
980
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
