CentOS7.9-安装k8s-1.20.2
所有主机设置域名解析
cat <<EOF | sudo tee -a /etc/hosts
10.0.17.100 master
10.0.17.101 node1
10.0.17.102 node2
EOF
ssh-keygen -f /root/.ssh/id_rsa -N ''
for i in node1 node2; do ssh-copy-id $i; done;
主机配置好对应主机名和解析
hostnamectl set-hostname master
hostnamectl set-hostname node1
hostnamectl set-hostname node2
禁用firewalld
systemctl disable firewalld --now
iptables -t nat -F && iptables -t mangle -F
安装容器运行时
所有节点安装Docker
# (安装 Docker CE)
## 设置仓库
### 安装所需包
sudo yum install -y yum-utils device-mapper-persistent-data lvm2
### 新增 Docker 仓库
sudo yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
## 安装 Docker CE
sudo yum update -y && sudo yum install -y \
containerd.io-1.2.13 \
docker-ce-19.03.11 \
docker-ce-cli-19.03.11
## 创建 /etc/docker 目录
sudo mkdir /etc/docker
# 设置 Docker daemon
cat <<EOF | sudo tee /etc/docker/daemon.json
{
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
},
"storage-driver": "overlay2",
"storage-opts": [
"overlay2.override_kernel_check=true"
],
"registry-mirrors": [
"https://docker.rainbond.cc",
"https://registry.docker-cn.com",
"http://hub-mirror.c.163.com"
]
}
EOF
# Create /etc/systemd/system/docker.service.d
sudo mkdir -p /etc/systemd/system/docker.service.d
# 启动 Docker
sudo systemctl daemon-reload
sudo systemctl restart docker
# 开机启动 docker
sudo systemctl enable docker --now
所有节点安装 kubeadm、kubelet 和 kubectl
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
# 将 SELinux 设置为 permissive 模式(相当于将其禁用)
setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
# 安装节点 kubelet kubeadm kubectl
yum install -y kubelet-1.20.9 kubeadm-1.20.9 kubectl-1.20.9
yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
# 默认配置的pause镜像使用gcr.io仓库,国内可能无法访问,所以这里配置Kubelet使用私有仓库的pause镜像:
cat >/etc/sysconfig/kubelet<<EOF
KUBELET_EXTRA_ARGS="--pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2"
EOF
systemctl daemon-reload
systemctl enable --now kubelet # (如果启动失败无需管理,初始化成功以后即可启动)
## 操作系统简单初始化
### 关闭swap(阿里云centos7.9 不需要执行)
```
swapoff -a && sysctl -w vm.swappiness=0
sed -ri '/^[^#]*swap/s@^@#@' /etc/fstab
```
### 加载 br_netfilter 模块,设置必要sysctl 参数
```
sudo modprobe br_netfilter
# 设置必需的 sysctl 参数,这些参数在重新启动后仍然存在。
cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF
# Apply sysctl params without reboot
sudo sysctl --system
```
# 配置时间同步
yum install -y ntpdate
ntpdate ntp.aliyun.com
### 重启系统
```
reboot
```
初始化控制平面节点
master节点执行初始化
安装命令行补齐工具
# 安装bash-completion
yum install bash-completion -y
# 生成kubectl 需要补齐的命令
kubectl completion bash
# 启用所有shell 会话中都引用 kubectl 自动补齐脚本
kubectl completion bash >/etc/bash_completion.d/kubectl
sudo kubeadm init --control-plane-endpoint=master --pod-network-cidr=10.224.0.0/16 --service-cidr=10.96.0.0/12 --apiserver-advertise-address=10.0.17.100 --kubernetes-version=v1.20.9 --image-repository registry.aliyuncs.com/google_containers
kubeadm init 首先运行一系列预检查以确保机器 准备运行 Kubernetes。这些预检查会显示警告并在错误时退出。然后 kubeadm init 下载并安装集群控制平面组件。这可能会需要几分钟。 完成之后你应该看到:
# --control-plane-endpoint master 指定控制平面节点的API服务器地址和端口,默认为自动检测的本地IP地址。
# --pod-network-cidr 指定Pod网络范围
# --service-cidr 指定服务网络的CIDR范围
# --apiserver-advertise-address= 指定API服务器广播的地址。默认为自动检测的本地IP地址
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 172.26.164.103:6443 --token 8p7lja.t64yo0w69fnvz6kf \
--discovery-token-ca-cert-hash sha256:517c42c7e689a50fd04ecb745c34899458ad06fe856d1d718caece81ec56b4e2
请运行以下命令,使用用户可以使用kubectl, 它们也是 kubeadm init 输出的一部分:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
记录 kubeadm init 输出的 kubeadm join 命令。 你需要此命令将节点加入集群。
重置 kubeadm 安装配置(初始化异常中断重装时使用)
rm -rf /etc/kubernetes/pki
kubeadm reset
添加node节点
在node1节点执行
kubeadm join 10.0.17.100:6443 --token 8p7lja.t64yo0w69fnvz6kf \
--discovery-token-ca-cert-hash sha256:517c42c7e689a50fd04ecb745c34899458ad06fe856d1d718caece81ec56b4e2
在master节点执行
kubectl get nodes
NAME STATUS ROLES AGE VERSION
master NotReady control-plane,master 36m v1.20.2
node1 NotReady <none> 12s v1.20.2
没有安装网络所以node状态是NotReady
安装calico
wget https://calico-v3-17.netlify.app/archive/v3.17/manifests/calico.yaml
vim calico.yaml
CALOCI_IPV4POOL_CIDR 需要修改
kubectl apply -f calico.yaml
kubectl 命令
查看
# 1 查看所有的ns
[root@master k8syamls]# kubectl get ns
NAME STATUS AGE
default Active 63m
dev Active 2m21s
kube-node-lease Active 63m
kube-public Active 63m
kube-system Active 63m
# 2 查看指定的ns 命令:kubectl get ns ns名称
[root@master k8syamls]# kubectl get ns dev
NAME STATUS AGE
dev Active 2m59s
# 3 指定输出格式 命令:kubectl get ns ns名称 -o 格式参数
# kubernetes支持的格式有很多,比较常见的是wide、json、yaml
[root@master k8syamls]# kubectl get ns dev -o yaml
apiVersion: v1
kind: Namespace
metadata:
creationTimestamp: "2024-08-16T07:23:04Z"
managedFields:
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:status:
f:phase: {}
manager: kubectl-create
operation: Update
time: "2024-08-16T07:23:04Z"
name: dev
resourceVersion: "6228"
uid: c6e24bdf-870e-416c-8241-62abdc2bc601
spec:
finalizers:
- kubernetes
status:
phase: Active
# 4 查看ns详情 命令:kubectl describe ns ns名称
[root@master k8syamls]# kubectl describe ns dev
Name: dev
Labels: <none>
Annotations: <none>
Status: Active
No resource quota.
namespace
创建删除
# 创建namespace
[root@master ~]# kubectl create ns dev
namespace/dev creat
# 删除namespace
[root@master ~]# kubectl delete ns dev
namespace "dev" deleteded
配置方式
首先准备一个yaml文件:ns-dev.yaml
vim ns-dev.yaml
apiVersion: v1
kind: Namespace
metadata:
name: dev
创建:kubectl create -f ns-dev.yaml
删除:kubectl delete -f ns-dev.yaml
POD
# 查看pod
[root@master k8syamls]# kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-5bd5c5d588-rmwrm 1/1 Running 0 47m
calico-node-59qkl 1/1 Running 0 47m
calico-node-6nw28 1/1 Running 0 47m
calico-node-fgksb 1/1 Running 0 47m
coredns-7f89b7bc75-wlsgw 1/1 Running 0 67m
coredns-7f89b7bc75-zh4pr 1/1 Running 0 67m
etcd-master 1/1 Running 1 67m
kube-apiserver-master 1/1 Running 1 67m
kube-controller-manager-master 1/1 Running 1 67m
kube-proxy-7n4ck 1/1 Running 1 67m
kube-proxy-c9cqb 1/1 Running 0 67m
kube-proxy-g4dsh 1/1 Running 0 67m
kube-scheduler-master 1/1 Running 1 67m
# 创建pod
# 命令格式: kubectl run (pod控制器名称) [参数]
# --image 指定Pod的镜像
# --port 指定端口
# --namespace 指定namespace
[root@master k8syamls]# kubectl run nginx --image=nginx:latest --port=80 --namespace dev
pod/nginx created
[root@master k8syamls]# kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
nginx 0/1 ContainerCreating 0 5s
[root@master k8syamls]# kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
nginx 1/1 Running 0 8m53s
#查看pod详细信息
[root@master k8syamls]# kubectl describe pod nginx -n dev
Name: nginx
Namespace: dev
Priority: 0
Node: node1/10.0.17.101
Start Time: Fri, 16 Aug 2024 15:23:33 +0800
Labels: run=nginx
Annotations: cni.projectcalico.org/podIP: 10.224.166.129/32
cni.projectcalico.org/podIPs: 10.224.166.129/32
Status: Running
IP: 10.224.166.129
IPs:
IP: 10.224.166.129
Containers:
nginx:
Container ID: docker://aca393efb4758006d5e547cdc6eb8fa01169b81de7ef44bca945c8773094ea2e
Image: nginx:latest
Image ID: docker-pullable://nginx@sha256:447a8665cc1dab95b1ca778e162215839ccbb9189104c79d7ec3a81e14577add
Port: 80/TCP
Host Port: 0/TCP
State: Running
Started: Fri, 16 Aug 2024 15:24:44 +0800
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-rh99r (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
default-token-rh99r:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-rh99r
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 9m35s default-scheduler Successfully assigned dev/nginx to node1
Normal Pulling 9m34s kubelet Pulling image "nginx:latest"
Normal Pulled 8m25s kubelet Successfully pulled image "nginx:latest" in 1m8.610978215s
Normal Created 8m25s kubelet Created container nginx
Normal Started 8m24s kubelet Started container nginx
# 获取podIP
[root@master k8syamls]# kubectl get pods -n dev -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx 1/1 Running 0 10m 10.224.166.129 node1 <none> <none>
# 访问nginx
[root@master k8syamls]# curl 10.224.166.129:80
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
# 删除指定Pod
[root@master ~]# kubectl delete pod nginx -n dev
pod "nginx" deleted
# 此时,显示删除Pod成功,但是再查询,发现又新产生了一个
[root@master ~]# kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
nginx 1/1 Running 0 21s
# 这是因为当前Pod是由Pod控制器创建的,控制器会监控Pod状况,一旦发现Pod死亡,会立即重建
# 此时要想删除Pod,必须删除Pod控制器
# 先来查询一下当前namespace下的Pod控制器
[root@master ~]# kubectl get deploy -n dev
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 1/1 1 1 9m7s
# 接下来,删除此PodPod控制器
[root@master ~]# kubectl delete deploy nginx -n dev
deployment.apps "nginx" deleted
# 稍等片刻,再查询Pod,发现Pod被删除了
[root@master ~]# kubectl get pods -n dev
No resources found in dev namespace.
YAML配置pod nginx
vim pod-nginx.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx
namespace: dev
spec:
containers:
- image: nginx:latest
name: pod
ports:
- name: nginx-port
containerPort: 80
protocol: TCP
# 创建:
kubectl create -f pod-nginx.yaml
# 删除:
kubectl delete -f pod-nginx.yaml
LABEL
Label是kubernetes系统中的一个重要概念。它的作用就是在资源上添加标识,用来对它们进行区分和选择。
Label的特点:
一个Label会以key/value键值对的形式附加到各种对象上,如Node、Pod、Service等等
一个资源对象可以定义任意数量的Label ,同一个Label也可以被添加到任意数量的资源对象上去
Label通常在资源对象定义时确定,当然也可以在对象创建后动态添加或者删除
可以通过Label实现资源的多维度分组,以便灵活、方便地进行资源分配、调度、配置、部署等管理工作。
一些常用的Label 示例如下:
版本标签:“version”:“release”, “version”:“stable”…
环境标签:“environment”:“dev”,“environment”:“test”,“environment”:“pro”
架构标签:“tier”:“frontend”,“tier”:“backend”
标签定义完毕之后,还要考虑到标签的选择,这就要使用到Label Selector,即:
Label用于给某个资源对象定义标识
Label Selector用于查询和筛选拥有某些标签的资源对象
当前有两种Label Selector:
基于等式的Label Selector
name = slave: 选择所有包含Label中key="name"且value="slave"的对象
env != production: 选择所有包括Label中的key="env"且value不等于"production"的对象
基于集合的Label Selector
name in (master, slave): 选择所有包含Label中的key="name"且value="master"或"slave"的对象
name not in (frontend): 选择所有包含Label中的key="name"且value不等于"frontend"的对象
标签的选择条件可以使用多个,此时将多个Label Selector进行组合,使用逗号","进行分隔即可。例如:
name=slave,env!=production
name not in (frontend),env!=production
命名方式
# 为pod资源打标签
[root@master ~]# kubectl label pod nginx-pod version=1.0 -n dev
pod/nginx-pod labeled
# 为pod资源更新标签
[root@master ~]# kubectl label pod nginx-pod version=2.0 -n dev --overwrite
pod/nginx-pod labeled
# 查看标签
[root@master ~]# kubectl get pod nginx-pod -n dev --show-labels
NAME READY STATUS RESTARTS AGE LABELS
nginx-pod 1/1 Running 0 10m version=2.0
# 筛选标签
[root@master ~]# kubectl get pod -n dev -l version=2.0 --show-labels
NAME READY STATUS RESTARTS AGE LABELS
nginx-pod 1/1 Running 0 17m version=2.0
[root@master ~]# kubectl get pod -n dev -l version!=2.0 --show-labels
No resources found in dev namespace.
#删除标签
[root@master ~]# kubectl label pod nginx-pod version- -n dev
pod/nginx-pod labeled
配置方式
apiVerion: v1
kind: Pod
metadata:
name: nginx
namespace: dev
labels:
version: "3.0"
env: "test"
spec:
containers:
- image: nginx:latest
name: pod
ports:
- name: nginx-port
containerPort: 80
protocol: TCP
Deployment
在kubernetes中,Pod是最小的控制单元,但是kubernetes很少直接控制Pod,一般都是通过Pod控制器来完成的。Pod控制器用于pod的管理,确保pod资源符合预期的状态,当pod的资源出现故障时,会尝试进行重启或重建pod。
在kubernetes中Pod控制器的种类有很多,本章节只介绍一种:Deployment。
命令
# 命令格式: kubectl create deployment 名称 [参数]
# --image 指定pod的镜像
# --port 指定端口
# --replicas 指定创建pod数量
# --namespace 指定namespace
[root@master ~]# kubectl create deployment nginx --image=nginx:latest --port=80 --replicas=3 -n dev
deployment.apps/nginx created
# 查看创建的Pod
[root@master ~]# kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
nginx-5ff7956ff6-6k8cb 1/1 Running 0 19s
nginx-5ff7956ff6-jxfjt 1/1 Running 0 19s
nginx-5ff7956ff6-v6jqw 1/1 Running 0 19s
# 查看deployment的信息
[root@master ~]# kubectl get deploy -n dev
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 3/3 3 3 2m42s
# UP-TO-DATE:成功升级的副本数量
# AVAILABLE:可用副本的数量
[root@master ~]# kubectl get deploy -n dev -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 3/3 3 3 2m51s nginx nginx:latest run=nginx
# 查看deployment的详细信息
[root@master ~]# kubectl describe deploy nginx -n dev
Name: nginx
Namespace: dev
CreationTimestamp: Wed, 08 May 2021 11:14:14 +0800
Labels: run=nginx
Annotations: deployment.kubernetes.io/revision: 1
Selector: run=nginx
Replicas: 3 desired | 3 updated | 3 total | 3 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: run=nginx
Containers:
nginx:
Image: nginx:latest
Port: 80/TCP
Host Port: 0/TCP
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
Progressing True NewReplicaSetAvailable
OldReplicaSets: <none>
NewReplicaSet: nginx-5ff7956ff6 (3/3 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 5m43s deployment-controller Scaled up replicaset nginx-5ff7956ff6 to 3
# 删除
[root@master ~]# kubectl delete deployment nginx -n dev
deployment.apps "nginx" deleted
配置操作
创建一个deploy-nginx.yaml,内容如下:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
run: nginx
template:
metadata:
labels:
run: nginx
spec:
containers:
- image: nginx:latest
name: nginx
ports:
- containerPort: 80
protocol: TCP
然后就可以执行对应的创建和删除命令了:
创建:kubectl create -f deploy-nginx.yaml
删除:kubectl delete -f deploy-nginx.yaml
Service
利用Deployment来创建一组Pod来提供具有高可用性的服务。
虽然每个Pod都会分配一个单独的Pod IP,然而却存在如下两问题:
Pod IP 会随着Pod的重建产生变化
Pod IP 仅仅是集群内可见的虚拟IP,外部无法访问
这样对于访问这个服务带来了难度。因此,kubernetes设计了Service来解决这个问题。
Service可以看作是一组同类Pod对外的访问接口。借助Service,应用可以方便地实现服务发现和负载均衡。
操作一:创建集群内部可访问的Service
# 暴露Service
[root@master ~]# kubectl expose deploy nginx --name=svc-nginx1 --type=ClusterIP --port=80 --target-port=80 -n dev
service/svc-nginx1 exposed
# 查看service
[root@master ~]# kubectl get svc svc-nginx1 -n dev -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
svc-nginx1 ClusterIP 10.109.179.231 <none> 80/TCP 3m51s run=nginx
# 这里产生了一个CLUSTER-IP,这就是service的IP,在Service的生命周期中,这个地址是不会变动的
# 可以通过这个IP访问当前service对应的POD
[root@master ~]# curl 10.109.179.231:80
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
</head>
<body>
<h1>Welcome to nginx!</h1>
.......
</body>
</html>
操作二:创建集群外部也可访问的Service
# 上面创建的Service的type类型为ClusterIP,这个ip地址只用集群内部可访问
# 如果需要创建外部也可以访问的Service,需要修改type为NodePort
[root@master ~]kubectl expose deployment nginx --name=svc-nginx2 --type=NodePort --port=80 --target-port=80 -n devev
service/svc-nginx2 exposed
# 此时查看,会发现出现了NodePort类型的Service,而且有一对Port(80:30211/TC)
[root@master k8syamls]# kubectl get svc svc-nginx2 -n dev -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
svc-nginx2 NodePort 10.108.6.182 <none> 80:30211/TCP 2m18s run=nginx
# 接下来就可以通过集群外的主机访问 节点IP:31928访问服务了
# 例如在的电脑主机上通过浏览器访问下面的地址
[root@master k8syamls]# curl 10.0.17.100:30211
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@master k8syamls]#
删除Service
kubectl delete svc svc-nginx-1 -n dev service "svc-nginx-1" deleted
配置方式
创建一个svc-nginx.yaml,内容如下:
apiVersion: v1
kind: Service
metadata:
name: svc-nginx
namespace: dev
spec:
clusterIP: 10.109.179.231 #固定svc的内网ip
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: ClusterIP
然后就可以执行对应的创建和删除命令了:
创建:kubectl create -f svc-nginx.yaml
删除:kubectl delete -f svc-nginx.yaml