Pod 污点和容忍度

Pod 污点和容忍度

给了节点选则的主动权，我们给节点打一个污点，不容忍的pod就运行不上来，污点就是定义在节点上的键值属性数据，可以定决定拒绝那些pod；
taints是键值数据，用在节点上，定义污点；
tolerations是键值数据，用在pod上，定义容忍度，能容忍哪些污点
pod亲和性是pod属性；但是污点是节点的属性，污点定义在nodeSelector上

1 查看

[root@master1 affinity]# kubectl describe nodes master1|grep Taints
Taints:             node-role.kubernetes.io/master:NoSchedule
[root@master1 affinity]# kubectl describe nodes node1|grep Taints
Taints:             <none>
[root@master1 affinity]# kubectl describe nodes node2|grep Taints
Taints:             <none>
#由上可以看出，默认情况下控制节点是有taints的，只允许系统pod，不允许业务pod, work节点是没有taints的，它允许所有pod调度

taints的effect用来定义对pod对象的排斥等级（效果）：

NoSchedule：

仅影响pod调度过程，当pod能容忍这个节点污点，就可以调度到当前节点，后来这个节点的污点改了，加了一个新的污点，使得之前调度的pod不能容忍了，那这个pod会怎么处理，对现存的pod对象不产生影响

---

NoExecute：

既影响调度过程，又影响现存的pod对象，如果现存的pod不能容忍节点后来加的污点，这个pod就会被驱逐

---

PreferNoSchedule：

最好不，也可以，是NoSchedule的柔性版本

---

在pod对象定义容忍度的时候支持两种操作：

1.等值密钥：key和value上完全匹配

2.存在性判断：key和effect必须同时匹配，value可以是空

在pod上定义的容忍度可能不止一个，在节点上定义的污点可能多个，需要琢个检查容忍度和污点能否匹配，每一个污点都能被容忍，才能完成调度，如果不能容忍怎么办，那就需要看pod的容忍度了

2 管理节点污点

[root@master1 affinity]# kubectl taint --help
Update the taints on one or more nodes.

  *  A taint consists of a key, value, and effect. As an argument here, it is expressed as key=value:effect.
  *  The key must begin with a letter or number, and may contain letters, numbers, hyphens, dots, and underscores, up to
253 characters.
  *  Optionally, the key can begin with a DNS subdomain prefix and a single '/', like example.com/my-app
  *  The value is optional. If given, it must begin with a letter or number, and may contain letters, numbers, hyphens,
dots, and underscores, up to  63 characters.
  *  The effect must be NoSchedule, PreferNoSchedule or NoExecute.
  *  Currently taint can only apply to node.

Examples:
  # Update node 'foo' with a taint with key 'dedicated' and value 'special-user' and effect 'NoSchedule'.
  # If a taint with that key and effect already exists, its value is replaced as specified.
  kubectl taint nodes foo dedicated=special-user:NoSchedule
  
  # Remove from node 'foo' the taint with key 'dedicated' and effect 'NoSchedule' if one exists.
  kubectl taint nodes foo dedicated:NoSchedule-
  
  # Remove from node 'foo' all the taints with key 'dedicated'
  kubectl taint nodes foo dedicated-
  
  # Add a taint with key 'dedicated' on nodes having label mylabel=X
  kubectl taint node -l myLabel=X  dedicated=foo:PreferNoSchedule
  
  # Add to node 'foo' a taint with key 'bar' and no value
  kubectl taint nodes foo bar:NoSchedule

Options:
      --all=false: Select all nodes in the cluster
      --allow-missing-template-keys=true: If true, ignore any errors in templates when a field or map key is missing in
the template. Only applies to golang and jsonpath output formats.
      --dry-run='none': Must be "none", "server", or "client". If client strategy, only print the object that would be
sent, without sending it. If server strategy, submit server-side request without persisting the resource.
      --field-manager='kubectl-taint': Name of the manager used to track field ownership.
  -o, --output='': Output format. One of:
json|yaml|name|go-template|go-template-file|template|templatefile|jsonpath|jsonpath-as-json|jsonpath-file.
      --overwrite=false: If true, allow taints to be overwritten, otherwise reject taint updates that overwrite existing
taints.
  -l, --selector='': Selector (label query) to filter on, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2)
      --template='': Template string or path to template file to use when -o=go-template, -o=go-template-file. The
template format is golang templates [http://golang.org/pkg/text/template/#pkg-overview].
      --validate=true: If true, use a schema to validate the input before sending it

Usage:
  kubectl taint NODE NAME KEY_1=VAL_1:TAINT_EFFECT_1 ... KEY_N=VAL_N:TAINT_EFFECT_N [options]

Use "kubectl options" for a list of global command-line options (applies to all commands).

例一： 把node2当成生产环境专用的，其它node节点当成test使用

[root@master1 affinity]# kubectl taint node node2 node-type=production:NoSchedule
node/node2 tainted
You have new mail in /var/spool/mail/root
[root@master1 affinity]# kubectl get pod -o wide
NAME                          READY   STATUS    RESTARTS   AGE    IP               NODE    NOMINATED NODE   READINESS GATES
nginx-test-75c685fdb7-8f7pc   1/1     Running   0          20h    10.244.166.137   node1   <none>           <none>
nginx-test-75c685fdb7-9j6ft   1/1     Running   0          20h    10.244.104.4     node2   <none>           <none>
pod-first                     1/1     Running   0          71m    10.244.166.143   node1   <none>           <none>
pod-node-affinity-demo        1/1     Running   0          102m   10.244.166.140   node1   <none>           <none>
pod-node-affinity-demo-2      1/1     Running   0          87m    10.244.104.5     node2   <none>           <none>
pod-second                    1/1     Running   1          71m    10.244.104.6     node2   <none>           <none>
[root@master1 affinity]# 

cat > pod-taint.yaml <<END
apiVersion: v1
kind: Pod
metadata:
  name: taint-pod
  namespace: default
  labels:
    tomcat: tomcat-pod
spec:
  containers:
  - name: taint-pod
    ports:
    - containerPort: 8080
    image: tomcat:8.5-jre8-alpine
    imagePullPolicy: IfNotPresent
END

kubectl apply -f pod-taint.yaml
[root@master1 taint]# kubectl get pods -o wide
NAME                          READY   STATUS    RESTARTS   AGE    IP               NODE    NOMINATED NODE   READINESS GATES
nginx-test-75c685fdb7-8f7pc   1/1     Running   0          20h    10.244.166.137   node1   <none>           <none>
nginx-test-75c685fdb7-9j6ft   1/1     Running   0          20h    10.244.104.4     node2   <none>           <none>
pod-first                     1/1     Running   0          79m    10.244.166.143   node1   <none>           <none>
pod-node-affinity-demo        1/1     Running   0          110m   10.244.166.140   node1   <none>           <none>
pod-node-affinity-demo-2      1/1     Running   0          94m    10.244.104.5     node2   <none>           <none>
pod-second                    1/1     Running   1          79m    10.244.104.6     node2   <none>           <none>
taint-pod                     1/1     Running   0          34s    10.244.166.144   node1   <none>           <none>

#由于pod-taint没有定义容忍度，不能容忍node2的污点，因此它不会被调度到node2上去，被调度到node1上去了

删除刚刚的pod

[root@master1 taint]# kubectl delete -f pod-taint.yaml
pod “taint-pod” deleted

node1也加一个污点

[root@master1 taint]# kubectl taint node node1 node-type=development:NoExecute
node/node1 tainted

[root@master1 taint]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-test-75c685fdb7-9j6ft 1/1 Running 0 20h 10.244.104.4 node2
nginx-test-75c685fdb7-zp2xg 0/1 Pending 0 46s
pod-node-affinity-demo-2 1/1 Running 0 101m 10.244.104.5 node2
pod-second 1/1 Running 1 86m 10.244.104.6 node2

可以看到node1节点上的pod会被驱逐

#
cat > pod-demo.yaml<<END
apiVersion: v1
kind: Pod
metadata:
  name: myapp-deploy
  namespace: default
  labels:
    app: myapp
    release: canary
spec:
      containers:
      - name: myapp
        image: ikubernetes/myapp:v1
        ports:
        - name: http
          containerPort: 80
      tolerations:
      - key: "node-type"
        operator: "Equal"
        value: "production"
        effect: "NoSchedule"
END

kubectl apply -f pod-demo.yaml
[root@master1 taint]# kubectl get pods -o wide
NAME                          READY   STATUS    RESTARTS   AGE     IP             NODE     NOMINATED NODE   READINESS GATES
myapp-deploy                  1/1     Running   0          7m56s   10.244.104.7   node2    <none>           <none>
nginx-test-75c685fdb7-9j6ft   1/1     Running   0          23h     10.244.104.4   node2    <none>           <none>
nginx-test-75c685fdb7-zp2xg   0/1     Pending   0          137m    <none>         <none>   <none>           <none>
pod-node-affinity-demo-2      1/1     Running   0          3h59m   10.244.104.5   node2    <none>           <none>
pod-second                    1/1     Running   3          3h43m   10.244.104.6   node2    <none>           <none>

删除节点污点

[root@master1 ~]# kubectl taint nodes node1 node-type=development:NoExecute-
node/node1 untainted
[root@master1 taint]# kubectl describe nodes node1|grep Taint
Taints:             <none>

[root@master1 ~]# kubectl taint nodes node2 node-type=production:NoSchedule-
node/node2 untainted
[root@master1 taint]# kubectl describe nodes node2|grep Taint
Taints:             <none>

aints:

[root@master1 ~]# kubectl taint nodes node2 node-type=production:NoSchedule-
node/node2 untainted
[root@master1 taint]# kubectl describe nodes node2|grep Taint
Taints: