节点亲和度
键值运算关系
- In: label的值在某个列表中
- NotIn: label的值不在某个列表中
- Gt: label的值大于某个值
- Lt: label的值小于某个值
- Exists:某个label存在
- DoesNotExist: 某个label不存在
pod.spec.nodeAffinity
- preferredDuringSchedulingIgnoredDuringExecution: 软策略
- requiredDuringSchedulingIgnoredDuringExecution: 硬策略
硬策略
apiVersion: v1
kind: Pod
metadata:
name: affinity
labels:
app: node-affinity-pod
spec:
containers:
- name: with-node-affinity
image: nginx:v1
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname ##node节点标签 kubectl get nodes --show-labels 查看节点标签
operator: NotIn/In ##不在node2运行/在node2运行
values:
- node2.example.com
---
[root@matser affinity]# kubectl create -f pod1.yaml
pod/affinity created
[root@matser affinity]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
affinity 1/1 Running 0 4s 10.244.1.114 node1.example.com <none> <none>
#In 必须在node2上运行, 如果没有node2节点就会处于pending状态
软策略
apiVersion: v1
kind: Pod
metadata:
name: affinity
labels:
app: node-affinity-pod
spec:
containers:
- name: with-node-affinity
image: nginx:v1
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
preference:
matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- node3.example.com
[root@matser affinity]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
affinity 1/1 Running 0 59s 10.244.1.115 node1.example.com <none> <none>
#我这里只有node1和node2. 软策略当node3 没有的时候也可以调度到其他节点
Pod 亲和性
pod.spec.affinity.PodAffinity/podAntAffinity
- preferredDuringSchedulingIgnoredDuringExecution: 软策略
- requiredDuringSchedulingIgnoredDuringExecution: 硬策略
PodAffinity 在同一个节点
kind: Pod
metadata:
name: pod-3
labels:
app: pod-3
spec:
containers:
- name: pod-3
image: nginx:v1
affinity:
podAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- node1
topologyKey: kubernetes.io/hostname
#如果有标签app=node1 ,那就必须和它在一起
[root@matser affinity]# kubectl get pod -o wide --show-labels
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES LABELS
affinity 1/1 Running 0 4m43s 10.244.1.116 node1.example.com <none> <none> app=node1
pod-3 1/1 Running 0 30s 10.244.1.117 node1.example.com <none> <none> app=pod-3
podAntAffinity 不在同一个节点
apiVersion: v1
kind: Pod
metadata:
name: pod-3
labels:
app: pod-3
spec:
containers:
- name: pod-3
image: nginx:v1
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- node1
topologyKey: kubernetes.io/hostname
#当标签有 app=node1 的时候,跟它不在同一个域
[root@matser affinity]# kubectl get pod -o wide --show-labelsNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES LABELS
affinity 1/1 Running 0 19m 10.244.1.116 node1.example.com <none> <none> app=node1
pod-3 1/1 Running 0 2m12s 10.244.2.16 node2.example.com <none> <none> app=pod-3
亲和性/反亲和性调度策略比较:
调度策略 | 匹配标签 | 操作标签 | 拓扑域支持 | 调度目标 |
---|---|---|---|---|
nodeAffinity | 主机 | In,NotIn,Exists,DoesNotExists,Gt,Lt | 否 | 指定主机 |
podAffinity | POD | In,NotIn,Exists,DoesNotExists | 是 | POD与指定POD同一拓扑域 |
podAntiAffinity | POD | In,NotIn,Exists,DoesNotExists | 是 | POD与指定POD不在同一拓扑域 |
Taint和Toleration
节点亲和性,是 pod 的一种属性(偏好或硬性要求),它使 pod 被吸引到一类的节点,Taint 则相反,它使节点额能够排斥一类特定的pod
Taint 和 Toleration 相互配合,可以用来避免pod 被分配到不合适的节点上。每个节点上都可以应用一个或多个taint 这表示对于那些不能容忍这些taint 的 pod ,是不会被该节点接受的。如果将toleration 应用与 pod 上,则表示这些pod 可以 (但不要求)被调度到具有匹配taint 的节点上。
污点(Taint)
Ⅰ、污点(Taint)
使用 kubectl taint
命令可以给某个Node节点设置污点,Node 被设置上污点之后就和 Pod 之间存在了一种相斥的关系,可以让 Node 拒绝 Pod 的调度执行,甚至将已经存在的 Pod 驱逐出去
每个污点的组成:
key=value:effect
每个污点有一个 key 和 value 作为污点的标签,其中value 可以为空, effect 藐视污点的作用,当前taint effect 支持如下三个选项:
- NoSchedule:表示k8s将不会将 Pod 调度到具有该污点的Node 上
- PreferNoSchedule:表示k8s将尽量避免将 Pod调度到具有该污点的 Node 上
- NoExecute: 表示k8s 不会将 Pod 调度到具有该污点的 Node 上,同时会将Node 上已经存在的 Pod 驱逐出去
Ⅱ、污点的设置、查看和去除
#设置污点
kubectl taint nodes node1 key1=value1:NoSchedule
#节点说明,查看 taints 字段
kubectl describe pod pod-name
#去除污点
kubectl taint nodes node1 key1:NoSchedule-
容忍(Tolerations)
设置了污点的 Node 将根据 taint 的effect:NoSchedule、PreferNoSchedule、NoExecute 和 Pod 之间产生互斥的关系, Pod将在一点程度上不会被调度到 Node 上,但我们可以在 Pod 上设置容忍(Toleration),意思是设置了容忍的 Pod 将可以容忍污点的存在,可以被调度到存在污点的 Node 上。
pod.spec.tolerations
tolerations:
- key: "key1"
operator: "Equal"
value: "value1"
effect: "NoSchedule"
tolerationSeconds: 3600
- key: "key1"
operator: "Equal"
value: "value1"
effect: "NoExecute"
- key: "key2"
operator: "Exists"
effect: "NoSchedule"
- 其中key,value,effect 要与 Node上设hi的taint 保持一致
- operator 的值为Exists 将会忽略value 值
- tolerationSeconds 用于描述当 Pod 需要被驱逐时可以在 Pod 上继续保留运行的时间
Ⅰ、当不指定key 值时,表示容忍所有污点eky:
tolerations:
- operator: "Exists"
Ⅱ、当不指定 effect 值时,表述容忍所有的污点作用
tolerations:
- key: "key"
operator: "Exists"
Ⅲ、有多个 Master 存在时,防止资源浪费,可以如下设置
kubectl taint nodes node-name node-role.kubernetes.io/master=:PreferNoSchedule
固定节点调度
指定调度节点
Ⅰ、Pod.spec.nodeName 将 Pod 直接调度到指定的 Node 节点上,会跳过 Schedule 的调度策略,该匹配规则是强制匹配
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: myweb
spec:
replicas: 7
template:
metadata:
labels:
app: myweb
spec:
nodeName: node2.example.com #匹配node2 的主机名,全部调度到node2 上。
containers:
- name: myweb
image: nginx:v1
ports:
- containerPort: 80
[root@matser affinity]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
affinity 1/1 Running 0 82m 10.244.1.116 node1.example.com <none> <none>
myweb-75cf949cb6-bt7s2 1/1 Running 0 14s 10.244.2.19 node2.example.com <none> <none>
myweb-75cf949cb6-lvhqh 1/1 Running 0 14s 10.244.2.20 node2.example.com <none> <none>
myweb-75cf949cb6-pxb8v 1/1 Running 0 14s 10.244.2.23 node2.example.com <none> <none>
myweb-75cf949cb6-sjjjm 1/1 Running 0 14s 10.244.2.22 node2.example.com <none> <none>
myweb-75cf949cb6-tqgl5 1/1 Running 0 14s 10.244.2.17 node2.example.com <none> <none>
myweb-75cf949cb6-ttbq9 1/1 Running 0 14s 10.244.2.18 node2.example.com <none> <none>
myweb-75cf949cb6-wqqg2 1/1 Running 0 14s 10.244.2.21 node2.example.com <none> <none>
pod-3 1/1 Running 0 65m 10.244.2.16 node2.example.com <none> <none>
Ⅱ、Pod.spec.nodeSelector: 通过kubernetes 的label-selector 机制选择节点,由调度器调度策略匹配label,而后调度 Pod 到目标节点,该匹配规则属于强制约束
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: myweb1111
spec:
replicas: 2
template:
metadata:
labels:
app: myweb11
spec:
nodeSelector:
cpucounts: two
containers:
- name: myweb11
image: nginx:v1
ports:
- containerPort: 80
[root@matser affinity]# kubectl get nodes --show-labels
NAME STATUS ROLES AGE VERSION LABELS
matser.example.com Ready master 18d v1.15.0 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=matser.example.com,kubernetes.io/os=linux,node-role.kubernetes.io/master=
node1.example.com Ready <none> 18d v1.15.0 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,cpucounts=two,kubernetes.io/arch=amd64,kubernetes.io/hostname=node1.example.com,kubernetes.io/os=linux
node2.example.com Ready <none> 16d v1.15.0 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=node2.example.com,kubernetes.io/os=linux
#可以看到由于只有node2节点上由cpucounts=two 的标签,所有pod 会全部调度到 node2上
[root@matser affinity]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
myweb1111-7dcc545cf6-8cf42 1/1 Running 0 55s 10.244.1.119 node1.example.com <none> <none>
myweb1111-7dcc545cf6-9lvdh 1/1 Running 0 55s 10.244.1.118 node1.example.com <none> <none>