一、Deployment
1. 运行Deployment
整体过程:
- 用户通过kubectl创建Deployment
- Deployment创建ReplicaSet
- ReplicaSet创建Pod
(1)运行一个deployment,命令将部署包含两个副本的Deployment nginx-deployment,容器镜像为nginx:1.7.9。
zy@k8s-master:~$ kubectl create deployment nginx-deployment --image=nginx:1.7.9 --replicas=2
deployment.apps/nginx-deployment created
(2)查看nginx-deployment状态,输出显示正常运行。
zy@k8s-master:~$ kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deployment 2/2 2 2 3m44s
(3) 使用kubectl describe deployments 查看更详细的信息。
zy@k8s-master:~$ kubectl describe deployments
Name: nginx-deployment
Namespace: default
CreationTimestamp: Thu, 16 Mar 2023 00:38:26 +0800
Labels: app=nginx-deployment
Annotations: deployment.kubernetes.io/revision: 1
Selector: app=nginx-deployment
Replicas: 2 desired | 2 updated | 2 total | 2 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: app=nginx-deployment
Containers:
nginx:
Image: nginx:1.7.9
Port: <none>
Host Port: <none>
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
Progressing True NewReplicaSetAvailable
OldReplicaSets: <none>
NewReplicaSet: nginx-deployment-84b896cf65 (2/2 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 5m15s deployment-controller Scaled up replica set nginx-deployment-84b896cf65 to 2
(4) kubectl get replicaset
验证了Deployment通过ReplicaSet来管理Pod的事实。
zy@k8s-master:~$ kubectl get replicaset
NAME DESIRED CURRENT READY AGE
nginx-deployment-84b896cf65 2 2 2 8m41s
(5)kubectl describe replicaset
通过这句话Controlled By: Deployment/nginx-deployment可以看出此ReplicaSet是由Deployment nginx-deployment创建的。
zy@k8s-master:~$ kubectl describe replicaset
Name: nginx-deployment-84b896cf65
Namespace: default
Selector: app=nginx-deployment,pod-template-hash=84b896cf65
Labels: app=nginx-deployment
pod-template-hash=84b896cf65
Annotations: deployment.kubernetes.io/desired-replicas: 2
deployment.kubernetes.io/max-replicas: 3
deployment.kubernetes.io/revision: 1
Controlled By: Deployment/nginx-deployment
Replicas: 2 current / 2 desired
Pods Status: 2 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
Labels: app=nginx-deployment
pod-template-hash=84b896cf65
Containers:
nginx:
Image: nginx:1.7.9
Port: <none>
Host Port: <none>
Environment: <none>
Mounts: <none>
Volumes: <none>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulCreate 12m replicaset-controller Created pod: nginx-deployment-84b896cf65-2stcz
Normal SuccessfulCreate 12m replicaset-controller Created pod: nginx-deployment-84b896cf65-vqmgh
(6)kubectl get pod
zy@k8s-master:~$ kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-deployment-84b896cf65-2stcz 1/1 Running 0 19m
nginx-deployment-84b896cf65-vqmgh 1/1 Running 0 19m
(7)kubectl descirbe pod
通过这句话Controlled By: ReplicaSet/nginx-deployment-84b896cf65指明Pod是由ReplicaSet nginx-deployment-84b896cf65创建的。Events记录了Pod的启动过程。
zy@k8s-master:~$ kubectl describe pod
Name: nginx-deployment-84b896cf65-2stcz
Namespace: default
Priority: 0
Node: k8s-node2/192.168.65.141
Start Time: Thu, 16 Mar 2023 00:38:26 +0800
Labels: app=nginx-deployment
pod-template-hash=84b896cf65
Annotations: <none>
Status: Running
IP: 10.244.2.3
IPs:
IP: 10.244.2.3
Controlled By: ReplicaSet/nginx-deployment-84b896cf65
Containers:
nginx:
Container ID: docker://44e382b4dda9b104dbd74ba7463a2111990fe1cba5eb070a99b9bd3f9b02ad1e
Image: nginx:1.7.9
Image ID: docker-pullable://nginx@sha256:e3456c851a152494c3e4ff5fcc26f240206abac0c9d794affb40e0714846c451
Port: <none>
Host Port: <none>
State: Running
Started: Thu, 16 Mar 2023 00:39:04 +0800
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-prt5v (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
default-token-prt5v:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-prt5v
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 21m default-scheduler Successfully assigned default/nginx-deployment-84b896cf65-2stcz to k8s-node2
Normal Pulling 21m kubelet Pulling image "nginx:1.7.9"
Normal Pulled 20m kubelet Successfully pulled image "nginx:1.7.9" in 37.234491722s
Normal Created 20m kubelet Created container nginx
Normal Started 20m kubelet Started container nginx
Name: nginx-deployment-84b896cf65-vqmgh
Namespace: default
Priority: 0
Node: k8s-node1/192.168.65.142
Start Time: Thu, 16 Mar 2023 00:38:26 +0800
Labels: app=nginx-deployment
pod-template-hash=84b896cf65
Annotations: <none>
Status: Running
IP: 10.244.1.5
IPs:
IP: 10.244.1.5
Controlled By: ReplicaSet/nginx-deployment-84b896cf65
Containers:
nginx:
Container ID: docker://9805e1debeeee0a6a2fc0ab1956694e99f43001fedf84d712539e149d3faaeab
Image: nginx:1.7.9
Image ID: docker-pullable://nginx@sha256:e3456c851a152494c3e4ff5fcc26f240206abac0c9d794affb40e0714846c451
Port: <none>
Host Port: <none>
State: Running
Started: Thu, 16 Mar 2023 00:39:03 +0800
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-prt5v (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
default-token-prt5v:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-prt5v
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 21m default-scheduler Successfully assigned default/nginx-deployment-84b896cf65-vqmgh to k8s-node1
Warning FailedMount 21m kubelet MountVolume.SetUp failed for volume "default-token-prt5v" : failed to sync secret cache: timed out waiting for the condition
Normal Pulling 21m kubelet Pulling image "nginx:1.7.9"
Normal Pulled 20m kubelet Successfully pulled image "nginx:1.7.9" in 34.475536479s
Normal Created 20m kubelet Created container nginx
Normal Started 20m kubelet Started container nginx
2. 命令 vs 配置文件
k8s支持两种创建资源的方式:
(1)用 kubectl 命令直接创建。
(2)通过配置文件和 kubectl apply 创建。
新建一个nginx.yml的文件如下:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:latest
ports:
- containerPort: 80
zy@k8s-master:~$ kubectl apply -f nginx.yml
deployment.apps/nginx-deployment created
kubectl apply不但能够创建k8s集群,也能对资源进行更新,非常方便。不过k8s还提供了几个类似的命令,例如kubectl create、kubectl replace、kubectl edit和 kubectl patch。尽量只使用kubectl apply。
删除deployment:
kubectl delete deployments.apps nginx-deployment
3. Deployment配置文件简介
以刚才的nginx.yml为例:
apiVersion: apps/v1:使用的 Kubernetes API 版本。
kind: Deployment:定义了一个 Deployment 对象。
metadata:定义 Deployment 的元数据信息,包括名称和标签。
spec:定义 Deployment 的规格信息,包括副本数、选择器、模板等。
replicas: 3:指定要创建的 Pod 副本数量为 3。
selector:定义了选择器,用于选择要管理的 Pod。
matchLabels:选择器的标签条件,这里使用 app: nginx,表示选择标签 app=nginx 的 Pod。
template:定义了要创建的 Pod 的模板。
metadata:模板的元数据信息,包括标签。
labels:定义 Pod 的标签。
containers:定义容器信息,包括容器名称、镜像、端口等。
name: nginx:定义容器名称为 nginx。
image: nginx:latest:定义容器使用的镜像为最新版本的 nginx。
ports:定义容器需要暴露的端口。
containerPort: 80:定义容器监听的端口为 80。
apiVersion: apps/v1 #当前配置格式的版本
kind: Deployment #要创建的资源类型
metadata: #该资源的元数据 name是必须的元数据项
name: nginx-deployment
labels:
app: nginx
spec: #spec是该Deployment的规格说明
replicas: 3 #副本数量
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:latest
ports:
- containerPort: 80
4. 伸缩
伸缩指在线增加或减少Pod的副本数。
nginx-deployment初始是3个副本,修改nginx.yml文件,将副本改成5个:
replicas: 5
再次执行kubectl apply:
zy@k8s-master:~$ kubectl apply -f nginx.yml
deployment.apps/nginx-deployment configured
查看Pod:
zy@k8s-master:~$ kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-deployment-585449566-7bwvc 1/1 Running 0 5m24s 10.244.1.8 k8s-node1 <none> <none>
nginx-deployment-585449566-bd2sv 1/1 Running 0 5m24s 10.244.2.6 k8s-node2 <none> <none>
nginx-deployment-585449566-ghqgr 1/1 Running 0 2m17s 10.244.1.10 k8s-node1 <none> <none>
nginx-deployment-585449566-lpsfr 1/1 Running 0 2m17s 10.244.1.9 k8s-node1 <none> <none>
nginx-deployment-585449566-skk9r 1/1 Running 0 5m24s 10.244.2.5 k8s-node2 <none> <none>
如果考虑将k8s-master也当作Node使用,可以执行:
kubectl taint node k8s-master node-role.kubernetes.io/master-
如果要恢复Master Only状态,执行如下命令:
kubectl taint node k8s-master node-role.kubernetes.io/master="":NoSchedule
将副本减少为两个,修改nginx.yml:
replicas: 2
只剩两个副本:
zy@k8s-master:~$ kubectl apply -f nginx.yml
deployment.apps/nginx-deployment configured
zy@k8s-master:~$ kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-deployment-585449566-bd2sv 1/1 Running 0 18m 10.244.2.6 k8s-node2 <none> <none>
nginx-deployment-585449566-skk9r 1/1 Running 0 18m 10.244.2.5 k8s-node2 <none> <none>
5. Faliover
模拟k8s-node2故障,关闭该节点:
zy@k8s-node2:~$ sudo halt -h
等待一段时间k8s会检查到k8s-node2不可用,将k8s-node2上的pod标记为终止状态,并在k8s-node1上创建两个新的pod。
zy@k8s-master:~$ kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-deployment-585449566-bd2sv 1/1 Terminating 0 26m 10.244.2.6 k8s-node2 <none> <none>
nginx-deployment-585449566-hx7kp 0/1 ContainerCreating 0 27s <none> k8s-node1 <none> <none>
nginx-deployment-585449566-skk9r 1/1 Terminating 0 26m 10.244.2.5 k8s-node2 <none> <none>
nginx-deployment-585449566-z54q5 1/1 Running 0 27s 10.244.1.11 k8s-node1 <none>
6. 用label控制Pod的位置
默认配置下,Scheduler会将Pod调度到所有可用的Node。不过有些情况我们希望将pod部署到指定的Node。
(1)标记k8s-node1是配置了SSD的节点:
zy@k8s-master:~$ kubectl label node k8s-node1 disktype=ssd
node/k8s-node1 labeled
(2)查看节点的label:
disktype=ssd已经成功添加到k8s-node1上
zy@k8s-master:~$ kubectl get node --show-labels
NAME STATUS ROLES AGE VERSION LABELS
k8s-master Ready master 17h v1.19.10 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-master,kubernetes.io/os=linux,node-role.kubernetes.io/master=
k8s-node1 Ready <none> 17h v1.19.10 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,disktype=ssd,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-node1,kubernetes.io/os=linux
k8s-node2 Ready <none> 17h v1.19.10 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-node2,kubernetes.io/os=linux
(3)将指定的pod部署到k8s-node1
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 4
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:latest
ports:
- containerPort: 80
nodeSelector: #指定类型
disktype: ssd
重新apply一下:
zy@k8s-master:~$ kubectl apply -f nginx.yml
deployment.apps/nginx-deployment configured
查看pod,可以看到全都跑在k8s-node1上了
zy@k8s-master:~$ kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-deployment-5b5ff6dc64-2xvbg 1/1 Running 0 2m31s 10.244.1.16 k8s-node1 <none> <none>
nginx-deployment-5b5ff6dc64-5zvdc 1/1 Running 0 3m5s 10.244.1.13 k8s-node1 <none> <none>
nginx-deployment-5b5ff6dc64-qlg4j 1/1 Running 0 3m5s 10.244.1.14 k8s-node1 <none> <none>
nginx-deployment-5b5ff6dc64-wdvcr 1/1 Running 0 2m47s 10.244.1.15 k8s-node1 <none> <none>
(4)删除label disktype -即删除
zy@k8s-master:~$ kubectl label node k8s-node1 disktype-
node/k8s-node1 labeled
二、DaemonSet
Deployment部署的副本Pod会分布在各个Node上,每个Node都可能运行好几个副本。DaemonSet的不同之处在于:每个Node上最多只能运行一个副本。
DaemonSet的典型应用场景:
(1)在集群的每个节点上运行存储Daemon,比如glusterd或ceph。
(2)在每个节点上运行日志收集Daemon,比如flunentd或logstash。
(3)在每个节点上运行监控Daemon,比如Prometheus Node Exporter或collectd。
k8s自己就在用DaemonSet运行系统组件,执行如下命令:
zy@k8s-master:~$ kubectl get daemonsets.apps --namespace=kube-system
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
kube-proxy 3 3 3 3 3 kubernetes.io/os=linux 18h
DaemonSet kube-flannel-ds 和 kube-proxy 分别负责在每个节点上运行flannel和kube-proxy组件:
zy@k8s-master:~$ kubectl get pod --namespace=kube-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coredns-6d56c8448f-fdgm2 0/1 Running 46 18h 10.244.0.2 k8s-master <none> <none>
coredns-6d56c8448f-mkgx2 0/1 CrashLoopBackOff 45 18h 10.244.0.3 k8s-master <none> <none>
etcd-k8s-master 1/1 Running 0 18h 192.168.65.145 k8s-master <none> <none>
kube-apiserver-k8s-master 1/1 Running 2 18h 192.168.65.145 k8s-master <none> <none>
kube-controller-manager-k8s-master 1/1 Running 3 18h 192.168.65.145 k8s-master <none> <none>
kube-proxy-9rj9s 1/1 Running 0 17h 192.168.65.142 k8s-node1 <none> <none>
kube-proxy-sm5kd 1/1 Running 0 18h 192.168.65.145 k8s-master <none> <none>
kube-proxy-vgzgz 1/1 Running 1 17h 192.168.65.141 k8s-node2 <none> <none>
kube-scheduler-k8s-master 1/1 Running 3 18h 192.168.65.145 k8s-master <none> <none>
三、Job
容器按照持续运行时间可以分为两类:服务类容器和工作类容器。
服务类容器需要一直运行,如http server、daemon等。工作类容器则是一次性任务,比如批处理程序,完成后容器就退出。
Kubernetes的Deployment、ReplicaSet和DaemonSet都用于管理服务类容器,对于工作类容器,用Job。
1. Pod成功情况
(1)写一个简单的Job配置文件 myjob.yml:
apiVersion: batch/v1
kind: Job
metadata:
name: myjob
spec:
template:
metadata:
name: myjob
spec:
containers:
- name: hello
image: busybox
command: ["echo","hello zy and k8s job!"]
restartPolicy: Never #对于job 只能设置Never
(2)apply
zy@k8s-master:~$ kubectl apply -f myjob.yml
job.batch/myjob created
(3)查看job状态
zy@k8s-master:~$ kubectl get pod
NAME READY STATUS RESTARTS AGE
myjob-pqn57 0/1 Completed 0 98s
(4)查看Pod的标准输出
zy@k8s-master:~$ kubectl logs myjob-pqn57
hello zy and k8s job!