目录
2.3 MetalLB 和 Ingress-Nginx 的搭配原理
3.1.6 实现 路径识别 ingress 控制器清单文件配置的解释
1 MetailLB 搭建
1.1 MetalLB 的作用和原理
-
提供外部 IP 地址：
- MetalLB 的主要作用是为 Kubernetes 集群中的服务提供外部可访问的 IP 地址。在没有云服务提供商提供负载均衡器的情况下，MetalLB 可以模拟实现类似功能。
- MetalLB 支持两种地址分配模式：二层模式 和 边界网关协议（BGP）模式。
- 二层模式：通过在局域网中广播地址解析协议（ARP）请求来宣告服务的 IP 地址，将流量引导到拥有该 IP 地址的节点上。
- BGP 模式：使用 BGP 协议与网络中的路由器进行通信，宣告服务的 IP 地址，并引导外部流量进入集群。
-
负载均衡流量：
- 在将流量引导到拥有服务 IP 地址的节点后，MetalLB 可以根据配置的策略将流量分发到不同的后端 Pod 上。
- 例如，在二层模式下，可以使用轮询或随机等方式进行流量分发。
MetalLB官网
https://metallb.universe.tf/installation/
1.2 MetalLB功能
为 LoadBalancer 分配 vip
LoadBalancer类型的Service
LoadBalancer和NodePort很相似，目的都是向外部暴露一个端口，区别在于LoadBalancer会在集群的外部再来做一个负载均衡设备，而这个设备需要外部环境支持的，外部服务发送到这个设备上的请求，会被设备负载之后转发到集群中。
1.3 部署 MetalLB
1.3.1 创建deployment控制器和创建一个服务
[root@k8s-master metalb]# kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3d14h
[root@k8s-master metalb]# kubectl create deployment dep \
--image nginx:latest \
--dry-run=client \
--port 80 --replicas 3 -o yaml > dep.yml
# 修改好的如下
[root@k8s-master metalb]# cat dep.yml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: dep
name: dep
spec:
replicas: 3
selector:
matchLabels:
app: dep
template:
metadata:
labels:
app: dep
spec:
containers:
- image: nginx:latest
name: nginx
ports:
- containerPort: 80
[root@k8s-master metalb]# kubectl apply -f dep.yml
[root@k8s-master metalb]# kubectl get pods
NAME READY STATUS RESTARTS AGE
dep-79fcdcdfc7-27qzq 1/1 Running 0 63s
dep-79fcdcdfc7-sjjzz 1/1 Running 0 63s
dep-79fcdcdfc7-x7rdz 1/1 Running 0 63s
# 此时还没有创建服务
[root@k8s-master metalb]# kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3d15h
# 创建服务
[root@k8s-master metalb]# kubectl expose deployment dep \
--name=svc-nginx \
--type=LoadBalancer \
--port=80 --target-port=80 \
--dry-run=client -o yaml >> dep.yml
# 修改之后
[root@k8s-master metalb]# cat dep.yml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: dep
name: dep
spec:
replicas: 3
selector:
matchLabels:
app: dep
template:
metadata:
labels:
app: dep
spec:
containers:
- image: nginx:latest
name: nginx
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
labels:
app: dep
name: svc-nginx
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: dep
type: LoadBalancer
[root@k8s-master metalb]# kubectl apply -f dep.yml
# 没有提供IP 因为是裸金属模式，需要借助插件来完成 如 MetalLB
[root@k8s-master metalb]# kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3d15h
svc-nginx LoadBalancer 10.106.13.221 <peding> 80/TCP 69m1.3.2 下载MealLB清单文件
[root@k8s-master metalb]# wget https://raw.githubusercontent.com/metallb/metallb/v0.14.8/config/manifests/metallb-native.yaml
1698 image: quay.io/metallb/controller:v0.14.8
1795 image: quay.io/metallb/speaker:v0.14.8
1.3.3 使用 docker 对镜像进行拉取
# 将镜像上传到私人仓库
[root@harbor harbor]# docker pull quay.io/metallb/controller:v0.14.8
[root@harbor harbor]# docker pull quay.io/metallb/speaker:v0.14.8
1.3.4 将镜像上传至私人仓库
[root@harbor ~]# docker login reg.shuyan.com
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
[root@harbor harbor]# docker tag registry.k8s.io/ingress-nginx/controller:v1.11.2 reg.shuyan.com/ingress-nginx/controller:v1.11.2
[root@harbor harbor]# docker push reg.shuyan.com/ingress-nginx/controller:v1.11.2
[root@harbor ~]# docker tag quay.io/metallb/speaker:v0.14.8 reg.shuyan.com/metallb/speaker:v0.14.8
[root@harbor ~]# docker push reg.shuyan.com/metallb/speaker:v0.14.8 1.3.5 将官方仓库地址修改为本地私人地址
[root@k8s-master metalb]# ls
metallb-native.yaml
[root@k8s-master metalb]# sed -i 's/quay.io\/metallb\/controller:v0.14.8/reg.shuyan.com\/metallb\/controller:v0.14.8/g' metallb-native.yaml
[root@k8s-master metalb]# sed -i 's/quay.io\/metallb\/speaker:v0.14.8/reg.shuyan.com\/metallb\/speaker:v0.14.8/g' metallb-native.yaml1.3.6 运行清单文件部署服务
[root@k8s-master metalb]# kubectl apply -f metallb-native.yaml
namespace/metallb-system created
customresourcedefinition.apiextensions.k8s.io/bfdprofiles.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bgpadvertisements.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bgppeers.metallb.io created
customresourcedefinition.apiextensions.k8s.io/communities.metallb.io created
customresourcedefinition.apiextensions.k8s.io/ipaddresspools.metallb.io created
customresourcedefinition.apiextensions.k8s.io/l2advertisements.metallb.io created
customresourcedefinition.apiextensions.k8s.io/servicel2statuses.metallb.io created
serviceaccount/controller created
serviceaccount/speaker created
role.rbac.authorization.k8s.io/controller created
role.rbac.authorization.k8s.io/pod-lister created
clusterrole.rbac.authorization.k8s.io/metallb-system:controller created
clusterrole.rbac.authorization.k8s.io/metallb-system:speaker created
rolebinding.rbac.authorization.k8s.io/controller created
rolebinding.rbac.authorization.k8s.io/pod-lister created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker created
configmap/metallb-excludel2 created
secret/metallb-webhook-cert created
service/metallb-webhook-service created
deployment.apps/controller created
daemonset.apps/speaker created
validatingwebhookconfiguration.admissionregistration.k8s.io/metallb-webhook-configuration created
# 查看命名空间是否建立
[root@k8s-master metalb]# kubectl get namespaces
NAME STATUS AGE
default Active 3d14h
dev Active 45h
kube-flannel Active 3d14h
kube-node-lease Active 3d14h
kube-public Active 3d14h
kube-system Active 3d14h
metallb-system Active 14s
# 查看镜像是否正确拉取
[root@k8s-master metalb]# kubectl -n metallb-system get pods
NAME READY STATUS RESTARTS AGE
controller-65957f77c8-mt8w8 1/1 Running 0 52s
speaker-f5znb 1/1 Running 0 52s
speaker-slsf7 1/1 Running 0 52s
speaker-wj79v 1/1 Running 0 52s1.3.7 配置 MetalLB 分配地址段
将以上官网的代码复制下来修改
[root@k8s-master metalb]# vim configmap.yml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: first-pool
namespace: metallb-system # 注意命名空间一定要和上面实体清单创建的一样
spec:
addresses:
- 192.168.239.240-192.168.239.250 # 注意此地址池一定要是本网段可用的地址
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: example
namespace: metallb-system # 注意命名空间一定要和上面实体清单创建的一样
spec:
ipAddressPools:
- first-pool声明地址池清单文件并访问测试
[root@k8s-master metalb]# kubectl apply -f configmap.yml
ipaddresspool.metallb.io/first-pool created
l2advertisement.metallb.io/example created
[root@k8s-master metalb]# kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3d15h
svc-nginx LoadBalancer 10.106.13.221 192.168.239.240 80:30668/TCP 12s
[root@k8s-master metalb]# curl 192.168.239.240
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>2 Ingress-nginx 原理及部署
Ingress-nginx官网https://kubernetes.github.io/ingress-nginx/deploy/
2.1 ingress-nginx 功能
-
一种全局的、为了代理不同后端 Service 而设置的负载均衡服务,支持7层
-
Ingress由两部分组成：Ingress controller和Ingress服务
-
Ingress Controller 会根据你定义的 Ingress 对象，提供对应的代理能力。
-
业界常用的各种反向代理项目，比如 Nginx、HAProxy、Envoy、Traefik 等，都已经为Kubernetes 专门维护了对应的 Ingress Controller。
2.2 Ingress-Nginx 的作用和原理
定义路由规则：
- Ingress-Nginx 是一个 Kubernetes Ingress 控制器，它根据 Ingress 资源定义的规则来路由外部 HTTP（S）流量到集群内的服务。
- Ingress 资源可以定义多个规则，每个规则可以指定一个主机名（如 example.com）和一个或多个路径（如 /path1 和 /path2），并将这些路径映射到后端服务。
反向代理和负载均衡：
- 当外部请求到达 Ingress-Nginx 控制器时，它作为反向代理将请求转发到相应的后端服务，具体是基于定义的规则来确定。
- Ingress-Nginx 可以实现负载均衡功能，将流量分发到多个后端 Pod 上。它支持多种负载均衡算法，如轮询、最少连接数等。
2.3 MetalLB 和 Ingress-Nginx 的搭配原理
部署 MetalLB：
- 在集群中部署 MetalLB，并通过配置来指定可用的 IP 地址池。这些 IP 地址将用于暴露集群内部的服务。
部署 Ingress-Nginx：
- 部署 Ingress-Nginx 控制器，通常会创建一个或多个服务（Service）来暴露 Ingress 控制器本身。这些服务可以配置为 NodePort 或者 LoadBalancer 类型。
- 由于在裸金属环境中可能没有 LoadBalancer 类型的支持，因此可以使用 MetalLB 来替代 LoadBalancer，将 Ingress-Nginx 控制器暴露给外部网络。
配置 Ingress 资源：
- 创建 Ingress 资源来定义 HTTP(S) 流量的规则。这些规则将告诉 Ingress-Nginx 如何处理来自外部的请求。
- Ingress 资源通常会引用前面创建的 Ingress-Nginx 控制器。
ingress 如何链接后端 service ：
1、修改服务类型
ingress 会创建自己的service 叫做 ingress-nginx-controller 修改 服务类型为 LoadBalancer
2、创建 ingress 资源类型：
在ingress的资源纪录类型中一定要注明service的名称否则无法正确转发
2.4 Ingress 部署
2.4.1 下载ingress-nginx yaml清单
[root@k8s-master metalb]# mkdir ingress
[root@k8s-master metalb]# cd ingress/
[root@k8s-master ingress]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.11.2/deploy/static/provider/aws/deploy.yaml
2.4.2 下载镜像并上传私有仓库
[root@k8s-master ingress]# vim deploy.yaml
451 image: registry.k8s.io/ingress-nginx/controller:v1.11.2
552 image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.3
[root@harbor ~]# docker pull registry.k8s.io/ingress-nginx/controller:v1.11.2
[root@harbor ~]# docker pull registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.3
[root@harbor ~]# docker tag registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.3 reg.shuyan.com/ingress-nginx/kube-webhook-certgen:v1.4.3
[root@harbor ~]# docker push reg.shuyan.com/ingress-nginx/kube-webhook-certgen:v1.4.3
[root@harbor harbor]# docker tag registry.k8s.io/ingress-nginx/controller:v1.11.2 reg.shuyan.com/ingress-nginx/controller:v1.11.2
[root@harbor harbor]# docker push reg.shuyan.com/ingress-nginx/controller:v1.11.22.4.3 修改清单镜像拉取地址
[root@k8s-master ingress]# ls
deploy.yaml
[root@k8s-master ingress]# sed -i 's/registry.k8s.io\/ingress-[root@k8s-master ingress]# nginx\/controller:v1.11.2/reg.shuyan.com\/ingress-nginx\/controller:v1.11.2/g' deploy.yaml
[root@k8s-master ingress]# sed -i 's/registry.k8s.io\/ingress-nginx\/kube-webhook-certgen:v1.4.3/reg.shuyan.com\/ingress-nginx\/kube-webhook-certgen:v1.4.3/g' deploy.yaml2.4.4 安装 Ingress-nginx
[root@k8s-master ingress]# kubectl apply -f deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
[root@k8s-master ingress]# kubectl get namespaces
NAME STATUS AGE
default Active 3d15h
dev Active 46h
ingress-nginx Active 37m
kube-flannel Active 3d15h
kube-node-lease Active 3d15h
kube-public Active 3d15h
kube-system Active 3d15h
metallb-system Active 62m2.5 测试 Ingress-nginx
2.5.1 查看是否正常并修改服务类型
[root@k8s-master ingress]# kubectl -n ingress-nginx get pods
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-dtnhp 0/1 Completed 0 40m
ingress-nginx-admission-patch-l9dp4 0/1 Completed 0 40m
ingress-nginx-controller-7d4db76476-hb9th 1/1 Running 0 40m
#修改微服务为loadbalancer
[root@k8s-master ~]# kubectl -n ingress-nginx edit svc ingress-nginx-controller
49 type: LoadBalancer
# 查看是否正确分配
[root@k8s-master ingress]# kubectl -n ingress-nginx get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.104.94.174 192.168.239.241 80:30654/TCP,443:32569/TCP 40m
ingress-nginx-controller-admission ClusterIP 10.104.152.104 <none> 443/TCP 40m2.5.2 创建 ingress 资源类型
[root@k8s-master ingress]# kubectl create ingress webcluster \
--rule '/=svc-nginx:80' \
--class nginx \
--dry-run=client -o yaml > ingress.yml
# 以下是修改过的文件
[root@k8s-master ingress]# cat ingress.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: webcluster
spec:
ingressClassName: nginx
rules:
- http:
paths:
- backend:
service:
name: svc-nginx
port:
number: 80
path: / # 这里指访问网站根的时候就会访问名为svc-nginx的这个服务
pathType: Prefix
# Exact（精确匹配），
# ImplementationSpecific（特定实现），
# Prefix（前缀匹配），
# Regular expression（正则表达式匹配）2.5.3 声明 ingress 资源类型
[root@k8s-master ingress]# kubectl apply -f ingress.yml
# 在此时svc-nginx 就不需要使用 LoadBlance 了可以换成ClusterIP实现后端pod负载均衡，
# ingress-nginx 使用 MetalLB 分配的地址，为自己使用，然后再将收到的数据传到后端service
# 有点像nginx的反向代理，流量先到 ingress-nginx 控制器再传到指定的 service
# 后端 service 不需要与外界通讯了自然就不需要用到 LoadBlance 去获得对外访问的IP了
# 只需要 ingress-nginx 对所有的 service 做一个管理，可以实现复杂的正则匹配。
# 修改名为 svc-nginx 的服务类型为 ClusterIP,从而实现后端各pod的负载均衡
[root@k8s-master metalb]# kubectl edit service svc-nginx
33 type: ClusterIP
# 检查是否改过来了
[root@k8s-master metalb]# kubectl get service svc-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
svc-nginx ClusterIP 10.106.13.221 <none> 80/TCP 6h50m2.5.4 测试 ingress-nginx 是否实现
[root@k8s-master metalb]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
dep-79fcdcdfc7-27qzq 1/1 Running 0 7h2m 10.244.2.51 k8s-node2 <none> <none>
dep-79fcdcdfc7-sjjzz 1/1 Running 0 7h2m 10.244.1.32 k8s-node1 <none> <none>
dep-79fcdcdfc7-x7rdz 1/1 Running 0 7h2m 10.244.2.52 k8s-node2 <none> <none>
[root@k8s-master metalb]# kubectl exec -it pods/dep-79fcdcdfc7-27qzq -- bash
root@dep-79fcdcdfc7-27qzq:/# echo this is `hostname -I` > /usr/share/nginx/html/index.html
[root@k8s-master metalb]# kubectl exec -it pods/dep-79fcdcdfc7-sjjzz -- bash
root@dep-79fcdcdfc7-sjjzz:/# echo this is `hostname -I` > /usr/share/nginx/html/index.html
[root@k8s-master metalb]# kubectl exec -it pods/dep-79fcdcdfc7-x7rdz -- bash
root@dep-79fcdcdfc7-x7rdz:/# echo this is `hostname -I` > /usr/share/nginx/html/index.html
[root@k8s-master metalb]# kubectl get service svc-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
svc-nginx ClusterIP 10.106.13.221 <none> 80/TCP 7h4m
[root@k8s-master metalb]# curl 10.106.13.221
this is 10.244.2.51
[root@k8s-master metalb]# curl 10.106.13.221
this is 10.244.1.32
[root@k8s-master metalb]# curl 10.106.13.221
this is 10.244.2.52
[root@k8s-master metalb]# kubectl -n ingress-nginx get service ingress-nginx-controller
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.104.94.174 192.168.239.241 80:30654/TCP,443:32569/TCP 6h53m
[root@k8s-master metalb]# curl 192.168.239.241
this is 10.244.2.51
[root@k8s-master metalb]# curl 192.168.239.241
this is 10.244.2.52
[root@k8s-master metalb]# curl 192.168.239.241
this is 10.244.1.322.5.5 回收资源
[root@k8s-master metalb]# cd ingress/
[root@k8s-master ingress]# ls
deploy.yaml ingress.yml
[root@k8s-master ingress]# cat ingress.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: webcluster
spec:
ingressClassName: nginx
rules:
- http:
paths:
- backend:
service:
name: svc-nginx
port:
number: 80
path: /
pathType: Prefix
[root@k8s-master ingress]# kubectl delete -f ingress.yml
[root@k8s-master ingress]# cd ..
[root@k8s-master metalb]# ls
configmap.yml dep.yml ingress metallb-native.yaml
[root@k8s-master metalb]# kubectl get deployments.apps dep
NAME READY UP-TO-DATE AVAILABLE AGE
dep 3/3 3 3 7h19m
[root@k8s-master metalb]# kubectl delete -f dep.yml
deployment.apps "dep" deleted
service "svc-nginx" deleted
[root@k8s-master metalb]# kubectl get deployments.apps
No resources found in default namespace.
[root@k8s-master metalb]# kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3d22h3 Ingress-nginx 的高级用法
3.1 基于路径的访问微服务
3.1.1 将 nginx 命名两个版本v1与v2
# 创建版本v1的deployment资源类型的nginx
[root@k8s-master ingress]# kubectl create deployment nginx-v1 \
--image nginx:latest \
--dry-run=client \
--port 80 \
--replicas 1 \
-o yaml > nginx-v1.yml
[root@k8s-master ingress]# cat nginx-v1.yml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx-v1 # 此标签一定要与微服务的标签对得上,不然微服务无法找到deployment
name: nginx-v1
spec:
replicas: 1
selector:
matchLabels:
app: nginx-v1
template:
metadata:
labels:
app: nginx-v1
spec:
containers:
- image: nginx:latest
name: nginx-v1
ports:
- containerPort: 80
# 创建版本 v2 的 deployment 资源类型的 nginx
[root@k8s-master ingress]# kubectl create deployment nginx-v2 \
--image nginx:latest \
--dry-run=client \
--port 80 \
--replicas 1 \
-o yaml > nginx-v2.yml
[root@k8s-master ingress]# cat nginx-v2.yml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx-v2
name: nginx-v2
spec:
replicas: 1
selector:
matchLabels:
app: nginx-v2
template:
metadata:
labels:
app: nginx-v2
spec:
containers:
- image: nginx:latest
name: nginx-v2
ports:
- containerPort: 80
# 声明这两个版本的清单文件
[root@k8s-master ingress]# kubectl apply -f nginx-v1.yml
deployment.apps/nginx-v1 created
[root@k8s-master ingress]# kubectl apply -f nginx-v2.yml
deployment.apps/nginx-v2 created
# 查看deployment是否正常运行
[root@k8s-master ingress]# kubectl get deployments.apps
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-v1 1/1 1 1 12s
nginx-v2 1/1 1 1 6s
3.1.2 暴露端口并指定微服务类型
创建微服务清单文件并将其加入到deployment的清单文件中
# 创建清单文件追加到deployment清单文件中
[root@k8s-master ingress]# kubectl expose deployment nginx-v1 \
--name=svc-nginx-v1 \
--port 80 --target-port 80 \
--dry-run=client \
--type=ClusterIP -o yaml >> nginx-v1.yml
[root@k8s-master ingress]# kubectl expose deployment nginx-v2 \
--name=svc-nginx-v2 --port 80 --target-port 80 \
--dry-run=client \
--type=ClusterIP -o yaml >> nginx-v2.yml
[root@k8s-master ingress]# cat nginx-v1.yml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx-v1
name: nginx-v1
spec:
replicas: 1
selector:
matchLabels:
app: nginx-v1
template:
metadata:
labels:
app: nginx-v1
spec:
containers:
- image: nginx:latest
name: nginx-v1
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
labels:
app: nginx-v1
name: svc-nginx-v1
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx-v1
type: ClusterIP
[root@k8s-master ingress]# cat nginx-v2.yml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx-v2
name: nginx-v2
spec:
replicas: 1
selector:
matchLabels:
app: nginx-v2
template:
metadata:
labels:
app: nginx-v2
spec:
containers:
- image: nginx:latest
name: nginx-v2
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
labels:
app: nginx-v2
name: svc-nginx-v2
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx-v2
type: ClusterIP
# 重新声明更新配置
[root@k8s-master ingress]# kubectl apply -f nginx-v1.yml
[root@k8s-master ingress]# kubectl apply -f nginx-v2.yml
# 服务创建成功
[root@k8s-master ingress]# kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3d22h
svc-nginx-v1 ClusterIP 10.107.76.175 <none> 80/TCP 15s
svc-nginx-v2 ClusterIP 10.100.188.171 <none> 80/TCP 9s3.1.3 进入 pod 修改默认发布文件
[root@k8s-master ingress]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-v1-dbd4bc45b-49hhw 1/1 Running 0 5m35s
nginx-v2-bd85b8bc4-nqpv2 1/1 Running 0 5m29s
[root@k8s-master ingress]# kubectl exec -it pods/nginx-v1-dbd4bc45b-49hhw -- bash
root@nginx-v1-dbd4bc45b-49hhw:/# echo this is nginx-v1 `hostname -I` > /usr/share/nginx/html/index.html
[root@k8s-master ingress]# kubectl exec -it pods/nginx-v2-bd85b8bc4-nqpv2 -- bash
root@nginx-v2-bd85b8bc4-nqpv2:/# echo this is nginx-v2 `hostname -I` > /usr/share/nginx/html/index.html 3.1.4 测试 service 是否正常
[root@k8s-master ingress]# kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3d22h
svc-nginx-v1 ClusterIP 10.107.76.175 <none> 80/TCP 15s
svc-nginx-v2 ClusterIP 10.100.188.171 <none> 80/TCP 9s
[root@k8s-master ingress]# curl 10.107.76.175
this is nginx-v1 10.244.2.54
[root@k8s-master ingress]# curl 10.100.188.171
this is nginx-v2 10.244.1.35创建七层负载
-- 基于路径识别访问哪个微服务
3.1.5 创建 ingress 资源类型
[root@k8s-master ingress]# kubectl create ingress webcluster \
--class nginx \
--rule "/v1=svc-nginx-v1:80" \
--rule "/v2=svc-nginx-v2:80" \
--dry-run=client -o yaml > ingress-route.yml 3.1.6 实现 路径识别 ingress 控制器清单文件配置的解释
[root@k8s-master ingress]# cat ingress-route.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: webcluster
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
# 由于在下面基于路径访问,实际传到后端服务的路径为 192.168.239.241/v1 或者 /v2
# 但是在后端nginx中默认发布路径中并没有这个目录，所以会无法找到。
# 所以就有了以上的配置 -- rewrite-target 重定向。
# 此条配置实现的效果：
# 比如说访问版本一按下面配置路径最终为192.168.239.241/v1/index.html
# 但加上rewrite-target: / 的这条配置 那么 Nginx Ingress 会将请求重写为
# 192.168.239.241/index.html
spec:
ingressClassName: nginx
rules:
- http:
paths:
- backend:
service:
name: svc-nginx-v1
port:
number: 80
path: /v1
pathType: Prefix
- backend:
service:
name: svc-nginx-v2
port:
number: 80
path: /v2
pathType: Prefix
# Exact（精确匹配），
# ImplementationSpecific（特定实现），
# Prefix（前缀匹配），
# Regular expression（正则表达式匹配）
在这个例子中，任何匹配 /v1 和 /v2 的请求都会被重写为新的目标路径 /，
然后转发到名为 svc-nginx-v1 和 svc-nginx-v2 的后端服务。3.1.7 声明 ingress 清单文件 并测试
# 声明创建ingress控制器
[root@k8s-master ingress]# kubectl apply -f ingress-route.yml
ingress.networking.k8s.io/webcluster created
# 查看ingress-nginx控制器是否正常
[root@k8s-master ingress]# kubectl -n ingress-nginx get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.104.94.174 192.168.239.241 80:30654/TCP,443:32569/TCP 7h30m
ingress-nginx-controller-admission ClusterIP 10.104.152.104 <none> 443/TCP 7h30m
# 查看分配的IP
[root@k8s-master ingress]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
webcluster nginx * 192.168.239.241 80 56s
# 测试版本是否正常访问
[root@k8s-master ingress]# curl 192.168.239.241/v1
this is nginx-v1 10.244.2.54
[root@k8s-master ingress]# curl 192.168.239.241/v2
this is nginx-v2 10.244.1.353.2 基于域名访问的微服务
在 3.1 的基础上做
3.2.1 创建 Ingress 资源类型
# 回收以上的ingress类型
[root@k8s-master ingress]# kubectl delete -f ingress-route.yml
# 注意创建ingress资源类型的时候 类必须为nginx 因为在ingress部署的时候类名就已经定好了
[root@k8s-master ingress]# kubectl get ingressclasses
NAME CONTROLLER PARAMETERS AGE
nginx k8s.io/ingress-nginx <none> 35h
# deploy.yml 为ingress的部署文件
[root@k8s-master ingress]# grep -A 9 Ingress deploy.yaml
kind: IngressClass
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.11.2
name: nginx
# 创建ingress的资源类型
[root@k8s-master ingress]# kubectl create ingress dum --class nginx \
--rule "nginxv1.shuyan.com/=svc-nginx-v1:80" \
--rule "nginxv2.shuyan.com/=svc-nginx-v2:80" \
--dry-run=client -o yaml > nginx-dum.yml
# 由于生成的文件还是与目标需求文件有些差异，下面是修改好的yaml文件
[root@k8s-master ingress]# cat nginx-dum.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: dum
spec:
ingressClassName: nginx
rules:
- host: nginxv1.shuyan.com
http:
paths:
- backend:
service:
name: svc-nginx-v1
port:
number: 80
path: /
pathType: Prefix
- host: nginxv2.shuyan.com
http:
paths:
- backend:
service:
name: svc-nginx-v2
port:
number: 80
path: /
pathType: Prefix3.2.2 声明并测试是否正常访问
[root@k8s-master ingress]# kubectl apply -f nginx-dum.yml
# 查看是否正确创建
[root@k8s-master ingress]# kubectl describe ingress dum
Name: dum
Labels: <none>
Namespace: default
Address: 192.168.239.241 # IP 有了证明成功了
Ingress Class: nginx
Default backend: <default>
Rules:
Host Path Backends
---- ---- --------
nginxv1.shuyan.com # 域名有了也证明成功了
/ svc-nginx-v1:80 (10.244.2.54:80)
nginxv2.shuyan.com
/ svc-nginx-v2:80 (10.244.1.35:80)
Annotations: <none>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 20m (x2 over 21m) nginx-ingress-controller Scheduled for sync
# 客户端做好域名解析
[root@harbor ~]# vim /etc/hosts
192.168.239.241 nginxv1.shuyan.com nginxv2.shuyan.com
# 测试是否成功
[root@harbor ~]# curl nginxv1.shuyan.com
this is nginx-v1 10.244.2.54
[root@harbor ~]# curl nginxv2.shuyan.com
this is nginx-v2 10.244.1.353.2.3 建立 tls 加密
创建 secret 加密类型
# 回收之前的ingress资源
[root@k8s-master ingress]# kubectl delete -f nginx-dum.yml
# 由于创建secret需要依靠证书来生成,所以得先有证书
[root@k8s-master tls]# yum install openssl
[root@k8s-master tls]# openssl req -newkey rsa:2048 \
-nodes -keyout tls.key \
-x509 -days 365 \
-subj "/CN=nginx-svc/O=nginx-svc" \
-out tls.crt
Generating a 2048 bit RSA private key
.......+++
...............................................................................................................+++
writing new private key to 'tls.key'
-----
# 创建secret使用tls加密方式,命名为web-tls-secret,并指定证书的私钥和证书的路径
[root@k8s-master tls]# kubectl create secret tls web-tls-secret \
--key /root/tls/tls.key \
--cert /root/tls/tls.crt
# 查看 secret 是否正确创建
[root@k8s-master tls]# kubectl get secrets
NAME TYPE DATA AGE
web-tls-secret kubernetes.io/tls 2 34m
[root@k8s-master tls]# kubectl describe secrets
Name: web-tls-secret
Namespace: default
Labels: <none>
Annotations: <none>
Type: kubernetes.io/tls
Data
====
tls.crt: 1147 bytes
tls.key: 1708 bytes创建Igress资源类型,添加所需的 secret 到 Igress资源清单中，使得最后运行能正确识别此secret
# 创建资源类型
[root@k8s-master tls]# kubectl create ingress tls \
--class nginx \
--rule "nginxv1.shuyan.com/=svc-nginx-v1:80" \
--rule "nginxv2.shuyan.com/=svc-nginx-v2:80" \
--dry-run=client -o yaml > tls.yml
[root@k8s-master tls]# cat tls.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: tls
spec:
# 增加了tls：以下的参数使得可以识别到 创建的secret
tls:
- hosts:
- nginxv1.shuyan.com
- nginxv2.shuyan.com
secretName: web-tls-secret
ingressClassName: nginx
rules:
- host: nginxv1.shuyan.com
http:
paths:
- backend:
service:
name: svc-nginx-v1
port:
number: 80
path: /
pathType: Prefix
- host: nginxv2.shuyan.com
http:
paths:
- backend:
service:
name: svc-nginx-v2
port:
number: 80
path: /
pathType: Prefix
3.2.4 建立 auth 认证
创建认证文件
[root@k8s-master auth]# yum install httpd-tools -y
[root@k8s-master auth]# htpasswd -bcm auth shuyan 123456
[root@k8s-master auth]# ls
auth
[root@k8s-master auth]# cat auth
shuyan:$apr1$Cqhl913B$Pexoaitb4OnILCdEZm/Kv0
建立 secret 并使用 generic 类型
[root@k8s-master auth]# kubectl create secret generic auth-web \
--from-file /root/auth/auth
[root@k8s-master auth]# kubectl describe secrets auth-web
Name: auth-web
Namespace: default
Labels: <none>
Annotations: <none>
Type: Opaque
Data
====
auth: 45 bytes创建 ingress 资源类型
[root@k8s-master auth]# kubectl create ingress auth \
> --class nginx \
> --rule "nginxv1.shuyan.com/=svc-nginx-v1:80" \
> --rule "nginxv2.shuyan.com/=svc-nginx-v2:80" \
> --dry-run=client -o yaml > auth.yml
# 以下是修改后的ingress资源清单
[root@k8s-master auth]# cat auth.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
# 增加以下三行
annotations:
nginx.ingress.kubernetes.io/auth-type: basic # 选择认证类型
nginx.ingress.kubernetes.io/auth-secret: auth-web # 选择 secret 的名字
nginx.ingress.kubernetes.io/auth-realm: "Please input username and password"
name: auth
spec:
ingressClassName: nginx
rules:
- host: nginxv1.shuyan.com
http:
paths:
- backend:
service:
name: svc-nginx-v1
port:
number: 80
path: /
pathType: Prefix
- host: nginxv2.shuyan.com
http:
paths:
- backend:
service:
name: svc-nginx-v2
port:
number: 80
path: /
pathType: Prefix
[root@k8s-master auth]# kubectl apply -f auth.yml
[root@k8s-master auth]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
auth nginx nginxv1.shuyan.com,nginxv2.shuyan.com 192.168.239.241 80 38s客户端测试测试是否成功
[root@harbor ~]# curl -k https://nginxv1.shuyan.com
<html>
<head><title>401 Authorization Required</title></head>
<body>
<center><h1>401 Authorization Required</h1></center>
<hr><center>nginx</center>
</body>
</html>
[root@harbor ~]# curl -k https://nginxv1.shuyan.com -ushuyan:123456
this is nginx-v1 10.244.2.54
[root@harbor ~]# curl -k https://nginxv2.shuyan.com -ushuyan:123456
this is nginx-v2 10.244.1.353.2.5 Igress 实现 rewrite 重定向
# 回收上面的镜像
[root@k8s-master auth]# kubectl delete -f auth.yml
# 查看 service 名字
[root@k8s-master auth]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 6d2h
svc-nginx-v1 ClusterIP 10.107.76.175 <none> 80/TCP 2d4h
svc-nginx-v2 ClusterIP 10.100.188.171 <none> 80/TCP 2d4h
# 创建资源类型
[root@k8s-master ingress-rewrite]# kubectl create ingress rewrite \
--class nginx \
--rule "nginxv1.shuyan.com/=svc-nginx-v1:80" \
--dry-run=client -o yaml > ingress-rewrite-app-root.yml
# 以下是修改过的配置,增加了几条参数
[root@k8s-master ingress-rewrite]# cat ingress-rewrite-app-root.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/app-root: /index.html # 指定根目录文件
name: rewrite
spec:
ingressClassName: nginx
rules:
- host: nginxv1.shuyan.com # 域名访问的ingress
http:
paths:
- backend:
service:
name: svc-nginx-v1 # 指定service名字
port:
number: 80
path: /
pathType: Prefix
[root@k8s-master ingress-rewrite]# kubectl apply -f ingress-rewrite-app-root.yml
[root@k8s-master ingress-rewrite]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
rewrite nginx nginxv1.shuyan.com 192.168.239.241 80 20s
测试是否成功访问
[root@harbor ~]# curl -L http://nginxv1.shuyan.com # 重定向
this is nginx-v1 10.244.2.54有一个问题就是假如中间惨咋着其他的目录他就会识别不到，为了解决这个问题，可以使用路径重定向
[root@harbor ~]# curl -L http://nginxv1.shuyan.com/shuyan/index.html
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.27.1</center>
</body>
</html>回收以上的资源类型
[root@k8s-master ingress-rewrite]# kubectl create ingress rewrite \
--class nginx \
--rule "nginxv1.shuyan.com/=svc-nginx-v1:80" \
--rule "nginxv2.shuyan.com/=svc-nginx-v2:80" \
--dry-run=client -o yaml > ingress-rewrite.yml
# 以下清单文件做了稍微的修改
[root@k8s-master ingress-rewrite]# cat ingress-rewrite.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: rewrite
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
ingressClassName: nginx
rules:
- host: nginxv1.shuyan.com
http:
paths:
- backend:
service:
name: svc-nginx-v1
port:
number: 80
path: /
pathType: Prefix
- host: nginxv2.shuyan.com
http:
paths:
- backend:
service:
name: svc-nginx-v2
port:
number: 80
path: /shuyan(/|$)(.*) # 正则匹配类型将/shuyan 结尾的 还有 /shuyan/ 的 还有/shuyan/index.html 都转换为 /index.html
pathType: ImplementationSpecific # 由于使用到正则匹配需要改变类型
# 声明并查看
[root@k8s-master ingress-rewrite]# kubectl apply -f ingress-rewrite.yml
[root@k8s-master ingress-rewrite]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
rewrite nginx nginxv1.shuyan.com,nginxv2.shuyan.com 192.168.239.241 80 8m53s测试重定向是否成功
[root@harbor ~]# curl http://nginxv2.shuyan.com/shuyan/index.html -L
this is nginx-v2 10.244.1.35
[root@harbor ~]# curl http://nginxv2.shuyan.com/shuyan -L
this is nginx-v2 10.244.1.35
[root@harbor ~]# curl http://nginxv2.shuyan.com/shuyan/ -L
this is nginx-v2 10.244.1.35
扫一扫
1979
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
