kubernetes中的微服务Service(ClusterIP、NodePort、LoadBalancer)

目录

一 、什么是微服务

二 、微服务的类型

三、 ipvs模式

3.1 ipvs模式配置方式

 四、微服务类型详解

4.1 clusterip

4.2 ClusterIP中的特殊模式headless

4.3 nodeport

4.4 loadbalancer

4.5 metalLB

4.6 externalname

五 Ingress-nginx

5.1 ingress-nginx功能

5.2 部署ingress

5.2.1 下载部署文件

5.2.2 安装ingress

5.2.3测试ingress

5.3 ingress 的高级用法

5.3.1 基于路径的访问

5.3.2 基于域名的访问

5.3.3 建立tls加密

5.3.4 建立auth认证

5.3.5 rewrite重定向

六 、Canary金丝雀发布

6.1 么是金丝雀发布

6.2 Canary发布方式

6.2.1 基于header(http包头)灰度

6.2.2 基于权重的灰度发布


一 、什么是微服务

用控制器来完成集群的工作负载,那么应用如何暴漏出去?需要通过微服务暴漏出去后才能被访问

  • Service是一组提供相同服务的Pod对外开放的接口。

  • 借助Service,应用可以实现服务发现和负载均衡。

  • service默认只支持4层负载均衡能力,没有7层功能。(可以通过Ingress实现)

 

 

二 、微服务的类型

微服务类型作用描述
ClusterIP默认值,k8s系统给service自动分配的虚拟IP,只能在集群内部访问
NodePort将Service通过指定的Node上的端口暴露给外部,访问任意一个NodeIP:nodePort都将路由到ClusterIP
LoadBalancer在NodePort的基础上,借助cloud provider创建一个外部的负载均衡器,并将请求转发到 NodeIP:NodePort,此模式只能在云服务器上使用
ExternalName将服务通过 DNS CNAME 记录方式转发到指定的域名(通过 spec.externlName 设定

示例:

#生成控制器文件并建立控制器
[root@k8s-master ~]# kubectl create deployment jcl--image myapp:v1  --replicas 2 --dry-run=client -o yaml > jcl.yaml

#生成微服务yaml追加到已有yaml中
[root@k8s-master ~]# kubectl expose deployment jcl--port 80 --target-port 80 --dry-run=client -o yaml >> jcl.yaml

[root@k8s-master ~]# vim jcl.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: jcl
  name: jcl
spec:
  replicas: 2
  selector:
    matchLabels:
      app: jcl
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: jcl
    spec:
      containers:
      - image: myapp:v1
        name: myapp
---										#不同资源间用---隔开

apiVersion: v1
kind: Service
metadata:
  labels:
    app: jcl
  name: jcl
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: jcl

[root@k8s-master ~]# kubectl apply  -f jcl.yaml
deployment.apps/jclcreated
service/jclcreated

[root@k8s-master ~]# kubectl get services
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP   19h
jcl          ClusterIP   10.99.127.134   <none>        80/TCP    16s

微服务默认使用iptables调度

[root@k8s-master ~]# kubectl get services  -o wide
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE    SELECTOR
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP   19h    <none>
jcl    ClusterIP   10.99.127.134   <none>        80/TCP    119s   app=jcl			#集群内部IP 134

#可以在火墙中查看到策略信息
[root@k8s-master ~]# iptables -t nat -nL
KUBE-SVC-I7WXYK76FWYNTTGM  6    --  0.0.0.0/0            10.99.127.134        /* default/jcl cluster IP */ tcp dpt:80

三、 ipvs模式

  • Service 是由 kube-proxy 组件,加上 iptables 来共同实现的

  • kube-proxy 通过 iptables 处理 Service 的过程,需要在宿主机上设置相当多的 iptables 规则,如果宿主机有大量的Pod,不断刷新iptables规则,会消耗大量的CPU资源

  • IPVS模式的service,可以使K8s集群支持更多量级的Pod

3.1 ipvs模式配置方式

1 在所有节点中安装ipvsadm

[root@k8s-所有节点 pod]yum install ipvsadm –y

2 修改master节点的代理配置

[root@k8s-master ~]# kubectl -n kube-system edit cm kube-proxy
    metricsBindAddress: ""
    mode: "ipvs"                            #设置kube-proxy使用ipvs模式
    nftables:

3 重启pod,在pod运行时配置文件中采用默认配置,当改变配置文件后已经运行的pod状态不会变化,所以要重启pod

[root@k8s-master ~]# kubectl -n kube-system get  pods   | awk '/kube-proxy/{system("kubectl -n kube-system delete pods "$1)}'

[root@k8s-master ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.96.0.1:443 rr
  -> 172.25.254.100:6443          Masq    1      0          0
TCP  10.96.0.10:53 rr
  -> 10.244.0.2:53                Masq    1      0          0
  -> 10.244.0.3:53                Masq    1      0          0
TCP  10.96.0.10:9153 rr
  -> 10.244.0.2:9153              Masq    1      0          0
  -> 10.244.0.3:9153              Masq    1      0          0
TCP  10.97.59.25:80 rr
  -> 10.244.1.17:80               Masq    1      0          0
  -> 10.244.2.13:80               Masq    1      0          0
UDP  10.96.0.10:53 rr
  -> 10.244.0.2:53                Masq    1      0          0
  -> 10.244.0.3:53                Masq    1      0          0

切换ipvs模式后,kube-proxy会在宿主机上添加一个虚拟网卡:kube-ipvs0,并分配所有service IP

[root@k8s-master ~]# ip a | tail
 inet6 fe80::c4fb:e9ff:feee:7d32/64 scope link
    valid_lft forever preferred_lft forever
8: kube-ipvs0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default
 link/ether fe:9f:c8:5d:a6:c8 brd ff:ff:ff:ff:ff:ff
 inet 10.96.0.10/32 scope global kube-ipvs0
    valid_lft forever preferred_lft forever
 inet 10.96.0.1/32 scope global kube-ipvs0
    valid_lft forever preferred_lft forever
 inet 10.97.59.25/32 scope global kube-ipvs0
    valid_lft forever preferred_lft forever

 四、微服务类型详解

4.1 clusterip

特点:

clusterip模式只能在集群内访问,并对集群内的pod提供健康检测和自动发现功能

示例:

[root@k8s-master Service]# vim clusterip.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: strongjcl
  name: strongjcl
spec:
  replicas: 2
  selector:
    matchLabels:
      app: strongjcl
  template:
    metadata:
      labels:
        app: strongjcl
    spec:
      containers:
      - image: k8s/myapp:v1
        name: myapp

---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: strongjcl
  name: strongjcl
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: strongjcl
  type: ClusterIP

[root@k8s-master Service]# kubectl apply -f clusterip.yml
deployment.apps/strongjcl unchanged
service/strongjcl configured
[root@k8s-master Service]# ip a | tail
    inet6 fe80::14cf:82ff:fe95:cd95/64 scope link
       valid_lft forever preferred_lft forever
9: kube-ipvs0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default
    link/ether 72:da:3e:d6:e5:e6 brd ff:ff:ff:ff:ff:ff
    inet 10.96.0.10/32 scope global kube-ipvs0
       valid_lft forever preferred_lft forever
    inet 10.96.0.1/32 scope global kube-ipvs0
       valid_lft forever preferred_lft forever
    inet 10.104.244.25/32 scope global kube-ipvs0
       valid_lft forever preferred_lft forever
service创建后集群DNS提供解析
[root@k8s-master Service]# dig jcl.default.svc.cluster.local @10.96.0.10

; <<>> DiG 9.16.23-RH <<>> jcl.default.svc.cluster.local @10.96.0.10
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 4584
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 1412ec4db7f77d88 (echoed)
;; QUESTION SECTION:
;jcl.default.svc.cluster.local. IN      A

;; AUTHORITY SECTION:
cluster.local.          30      IN      SOA     ns.dns.cluster.local. hostmaster.cluster.local. 1725606452 7200 1800 86400 30

;; Query time: 4 msec
;; SERVER: 10.96.0.10#53(10.96.0.10)
;; WHEN: Fri Sep 06 16:00:22 CST 2024
;; MSG SIZE  rcvd: 163

4.2 ClusterIP中的特殊模式headless

headless(无头服务)

对于无头 Services 并不会分配 Cluster IP,kube-proxy不会处理它们, 而且平台也不会为它们进行负载均衡和路由,集群访问通过dns解析直接指向到业务pod上的IP,所有的调度有dns单独完成

[root@k8s-master Service]# vim clusterip-headless.yml
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: strongjcl
  name: strongjcl
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: strongjcl
  type: ClusterIP
  clusterIP: None
[root@k8s-master Service]# kubectl apply -f clusterip-headless.yml
service/timinglee created
[root@k8s-master Service]# dig jcl.default.svc.cluster.local @10.96.0.10

; <<>> DiG 9.16.23-RH <<>> jcl.default.svc.cluster.local @10.96.0.10
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 19006
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: e26d47f0623eebf5 (echoed)
;; QUESTION SECTION:
;jcl.default.svc.cluster.local. IN      A

;; AUTHORITY SECTION:
cluster.local.          30      IN      SOA     ns.dns.cluster.local. hostmaster.cluster.local. 1725609975 7200 1800 86400 30

;; Query time: 0 msec
;; SERVER: 10.96.0.10#53(10.96.0.10)
;; WHEN: Fri Sep 06 16:07:37 CST 2024
;; MSG SIZE  rcvd: 163
#开启一个busyboxplus的pod测试
[root@k8s-master Service]# kubectl run  test --image k8s/busyboxplus -it
If you don't see a command prompt, try pressing enter.
/ # nslookup strongjcl
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name:      strongjcl
Address 1: 10.244.2.2 10-244-2-2.strongjcl.default.svc.cluster.local
Address 2: 10.244.1.2 10-244-1-2.strongjcl.default.svc.cluster.local
/ # curl strongjcl
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
/ # curl strongjcl/hostname.html
strongjcl-5cbbb8f5f4-zkfmx

4.3 nodeport

通过ipvs暴漏端口从而使外部主机通过master节点的对外ip:<port>来访问pod业务

其访问过程为:

[root@k8s-master Service]# vim nodeport.yml
apiVersion: v1
kind: Service
metadata:
  labels:
    app: strongjcl-service
  name: strongjcl-service
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: strongjcl
  type: NodePort

[root@k8s-master Service]# kubectl apply -f nodeport.yml
service/strongjcl-service created
[root@k8s-master Service]# kubectl get service strongjcl-service
NAME                TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
strongjcl-service   NodePort   10.104.102.186   <none>        80:32746/TCP   21s
nodeport在集群节点上绑定端口,一个端口对应一个服务
[root@k8s-master Service]# for i in {1..5}
> do
> curl 172.25.254.100:32746/hostname.html
> done
strongjcl-5cbbb8f5f4-zkfmx
strongjcl-5cbbb8f5f4-bwx5t
strongjcl-5cbbb8f5f4-zkfmx
strongjcl-5cbbb8f5f4-bwx5t
strongjcl-5cbbb8f5f4-zkfmx
> nodeport默认端口
>
> nodeport默认端口是30000-32767,超出会报错

[root@k8s-master Service]# vim nodeport.yml
apiVersion: v1
kind: Service
metadata:
  labels:
    app: strongjcl-service
  name: strongjcl-service
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
    nodePort: 33333
  selector:
    app: strongjcl
  type: NodePort

[root@k8s-master Service]# kubectl apply -f nodeport.yml
The Service "strongjcl-service" is invalid: spec.ports[0].nodePort: Invalid value: 33333: provided port is not in the valid range. The range of valid ports is 30000-32767
如果需要使用这个范围以外的端口就需要特殊设定
[root@k8s-master Service]# vim /etc/kubernetes/manifests/kube-apiserver.yaml
- --service-node-port-range=30000-40000
添加“--service-node-port-range=“ 参数,端口范围可以自定义

修改后api-server会自动重启,等apiserver正常启动后才能操作集群

集群重启自动完成在修改完参数后全程不需要人为干预
[root@k8s-master Service]# kubectl apply -f nodeport.yml
service/strongjcl-service created
[root@k8s-master Service]# kubectl get service strongjcl-service
NAME                TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
strongjcl-service   NodePort   10.96.111.139   <none>        80:33333/TCP   28s

[root@k8s-master Service]# kubectl delete -f nodeport.yml
[root@k8s-master Service]# kubectl apply -f nodeport.yml
service/strongjcl-service created
[root@k8s-master Service]# kubectl get service strongjcl-service
NAME                TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
strongjcl-service   NodePort   10.96.111.139   <none>        80:33333/TCP   28s

4.4 loadbalancer

云平台会为我们分配vip并实现访问,如果是裸金属主机那么需要metallb来实现ip的分配

[root@k8s-master Service]# vim loadbalancer.yml
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: strongjcl-service
  name: strongjcl-service
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: strongjcl
  type: LoadBalancer

[root@k8s-master Service]# kubectl apply -f loadbalancer.yml
service/strongjcl-service created
默认无法分配外部访问IP
[root@k8s-master Service]# kubectl get svc
NAME                TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
kubernetes          ClusterIP      10.96.0.1       <none>        443/TCP        140m
strongjcl-service   LoadBalancer   10.100.99.110   <pending>     80:34710/TCP   12s

 

4.5 metalLB

metalLB功能

为LoadBalancer分配vip

部署方式

1.设置ipvs模式
[root@k8s-master Service]# kubectl edit cm -n kube-system kube-proxy
mode: "ipvs"
ipvs:
  strictARP: true

[root@k8s-master Service]# kubectl -n kube-system get  pods   | awk '/kube-proxy/{system("kubectl -n kube-system delete pods "$1)}'
pod "kube-proxy-5kqbw" deleted
pod "kube-proxy-h8clb" deleted
pod "kube-proxy-wsr27" deleted
2.下载部署文件
3.修改文件中镜像地址,与harbor仓库路径保持一致
[root@k8s-master Service]# vim /root/metallb-native.yaml
...
image: metallb/controller:v0.14.8
image: metallb/speaker:v0.14.8
4.上传镜像到harbor
[root@k8s-master Service]# docker pull quay.io/metallb/controller:v0.14.8
[root@k8s-master Service]# docker pull quay.io/metallb/speaker:v0.14.8
[root@k8s-master Service]# docker tag quay.io/metallb/speaker:v0.14.8 reg.strongjcl.org/metallb/speaker:v0.14.8
[root@k8s-master Service]# docker tag quay.io/metallb/controller:v0.14.8 reg.strongjcl.org/metallb/controller:v0.14.8
[root@k8s-master Service]# docker push reg.strongjcl.org/metallb/speaker:v0.14.8
[root@k8s-master Service]# docker push reg.strongjcl.org/metallb/controller:v0.14.8
部署服务
[root@k8s-master Service]# kubectl apply -f metallb-native.yaml
namespace/metallb-system created
customresourcedefinition.apiextensions.k8s.io/bfdprofiles.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bgpadvertisements.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bgppeers.metallb.io created
customresourcedefinition.apiextensions.k8s.io/communities.metallb.io created
customresourcedefinition.apiextensions.k8s.io/ipaddresspools.metallb.io created
customresourcedefinition.apiextensions.k8s.io/l2advertisements.metallb.io created
customresourcedefinition.apiextensions.k8s.io/servicel2statuses.metallb.io created
serviceaccount/controller created
serviceaccount/speaker created
role.rbac.authorization.k8s.io/controller created
role.rbac.authorization.k8s.io/pod-lister created
clusterrole.rbac.authorization.k8s.io/metallb-system:controller created
clusterrole.rbac.authorization.k8s.io/metallb-system:speaker created
rolebinding.rbac.authorization.k8s.io/controller created
rolebinding.rbac.authorization.k8s.io/pod-lister created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker created
configmap/metallb-excludel2 created
secret/metallb-webhook-cert created
service/metallb-webhook-service created
deployment.apps/controller created
daemonset.apps/speaker created
validatingwebhookconfiguration.admissionregistration.k8s.io/metallb-webhook-configuration created

[root@k8s-master Service]# kubectl -n metallb-system get pods
NAME                          READY   STATUS    RESTARTS   AGE
controller-65957f77c8-9ssxg   1/1     Running   0          23s
speaker-k9l7f                 0/1     Running   0          23s
speaker-n4gs9                 0/1     Running   0          23s
speaker-xmhrx                 1/1     Running   0          23s
配置分配地址段
[root@k8s-master Service]# vim configmap.yml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: first-pool                                              #地址池名称
  namespace: metallb-system
spec:
  addresses:
  - 172.25.254.50-172.25.254.99                 #修改为自己本地地址段

---                                                                             #两个不同的kind中间必须加分割
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: example
  namespace: metallb-system
spec:
  ipAddressPools:
  - first-pool                                                  #使用地址池

[root@k8s-master Service]# kubectl apply -f configmap.yml
ipaddresspool.metallb.io/first-pool created
l2advertisement.metallb.io/example created
[root@k8s-master Service]# kubectl get services
NAME                TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)        AGE
kubernetes          ClusterIP      10.96.0.1       <none>          443/TCP        168m
strongjcl-service   LoadBalancer   10.100.99.110   172.25.254.50   80:34710/TCP   28m
#通过分配地址从集群外访问服务
[root@k8s-master Service]# curl 172.25.254.50
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>

4.6 externalname

  • 开启services后,不会被分配IP,而是用dns解析CNAME固定域名来解决ip变化问题

  • 一般应用于外部业务和pod沟通或外部业务迁移到pod内时

  • 在应用向集群迁移过程中,externalname在过度阶段就可以起作用了。

  • 集群外的资源迁移到集群时,在迁移的过程中ip可能会变化,但是域名+dns解析能完美解决此问题

示例:

[root@k8s-master Service]# vim externalname.yml
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: strongjcl-service
  name: strongjcl-service
spec:
  selector:
    app: strongjcl
  type: ExternalName
  externalName: www.strongjcl.org

[root@k8s-master Service]# kubectl apply  -f externalname.yml
service/strongjcl-service created
[root@k8s-master Service]# kubectl get service strongjcl-service
NAME                TYPE           CLUSTER-IP   EXTERNAL-IP         PORT(S)   AGE
strongjcl-service   ExternalName   <none>       www.strongjcl.org   <none>    13s

五 Ingress-nginx

官网:

Installation Guide - Ingress-Nginx Controller

5.1 ingress-nginx功能

  • 一种全局的、为了代理不同后端 Service 而设置的负载均衡服务,支持7层

  • Ingress由两部分组成:Ingress controller和Ingress服务

  • Ingress Controller 会根据你定义的 Ingress 对象,提供对应的代理能力。

  • 业界常用的各种反向代理项目,比如 Nginx、HAProxy、Envoy、Traefik 等,都已经为Kubernetes 专门维护了对应的 Ingress Controller。

5.2 部署ingress

5.2.1 下载部署文件

 [root@k8s-master ~]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.11.2/deploy/static/provider/baremetal/deploy.yaml

上传ingress所需镜像到harbor

[root@k8s-master ~]# docker tag registry.k8s.io/ingress-nginx/controller:v1.11.2@sha256:d5f8217feeac4887cb1ed21f27c2674e58be06bd8f5184cacea2a69abaf78dce reg.timinglee.org/ingress-nginx/controller:v1.11.2

[root@k8s-master ~]# docker tag registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.3@sha256:a320a50cc91bd15fd2d6fa6de58bd98c1bd64b9a6f926ce23a600d87043455a3 reg.timinglee.org/ingress-nginx/kube-webhook-certgen:v1.4.3

[root@k8s-master ~]# docker push reg.timinglee.org/ingress-nginx/controller:v1.11.2
[root@k8s-master ~]# docker push reg.timinglee.org/ingress-nginx/kube-webhook-certgen:v1.4.3

5.2.2 安装ingress

[root@k8s-master Service]# vim deploy.yaml
445         image: ingress-nginx/controller:v1.11.2
546         image: ingress-nginx/kube-webhook-certgen:v1.4.3
599         image: ingress-nginx/kube-webhook-certgen:v1.4.3

[root@k8s-master Service]# kubectl apply -f deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
[root@k8s-master Service]# kubectl -n ingress-nginx get pods
NAME                                       READY   STATUS              RESTARTS   AGE
ingress-nginx-admission-create-jjx9p       0/1     Completed           0          5s
ingress-nginx-admission-patch-c72mp        0/1     Completed           1          5s
ingress-nginx-controller-bb7d8f97c-p9x87   0/1     ContainerCreating   0          5s
[root@k8s-master Service]# kubectl -n ingress-nginx get svc
NAME                                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.110.177.205   <none>        80:33961/TCP,443:31071/TCP   19s
ingress-nginx-controller-admission   ClusterIP   10.96.22.40      <none>        443/TCP                      18s
#修改微服务为loadbalancer
[root@k8s-master Service]# kubectl -n ingress-nginx edit svc ingress-nginx-controller
49   type: LoadBalancer
[root@k8s-master Service]# kubectl -n ingress-nginx get services
NAME                                 TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)                      AGE
ingress-nginx-controller             LoadBalancer   10.110.177.205   172.25.254.50   80:33961/TCP,443:31071/TCP   113s
ingress-nginx-controller-admission   ClusterIP      10.96.22.40      <none>          443/TCP                      112s

5.2.3测试ingress

#生成yaml文件
[root@k8s-master Service]# kubectl create ingress webcluster --rule '*/=strongjcl-svc:80' --dry-run=client -o yaml > strongjcl-ingress.yml
[root@k8s-master Service]# vim strongjcl-ingress.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  creationTimestamp: null
  name: webcluster
spec:
  rules:
  - http:
      paths:
      - backend:
          service:
            name: strongjcl-svc
            port:
              number: 80
        path: /
        pathType: Prefix
#Exact(精确匹配),ImplementationSpecific(特定实现),Prefix(前缀匹配),Regular expression(正则表达式匹配)
#建立ingress控制器
[root@k8s-master Service]# kubectl apply -f strongjcl-ingress.yml
ingress.networking.k8s.io/webcluster created
[root@k8s-master Service]# kubectl get ingress
NAME           CLASS   HOSTS   ADDRESS         PORTS   AGE
test-ingress   nginx   *       172.25.254.10   80      76s

ingress必须和输出的service资源处于同一namespace

5.3 ingress 的高级用法

5.3.1 基于路径的访问

1.建立用于测试的控制器myapp

[root@k8s-master app]# kubectl create deployment myapp-v1 --image myapp:v1 --dry-run=client -o yaml > myapp-v1.yaml

[root@k8s-master app]# kubectl create deployment myapp-v2 --image myapp:v2 --dry-run=client -o yaml > myapp-v2.yaml


[root@k8s-master app]# vim myapp-v1.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: myapp-v1
  name: myapp-v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: myapp-v1
  strategy: {}
  template:
    metadata:
      labels:
        app: myapp-v1
    spec:
      containers:
      - image: myapp:v1
        name: myapp

---

apiVersion: v1
kind: Service
metadata:
  labels:
    app: myapp-v1
  name: myapp-v1
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: myapp-v1


[root@k8s-master app]# vim myapp-v2.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: myapp-v2
  name: myapp-v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: myapp-v2
  template:
    metadata:
      labels:
        app: myapp-v2
    spec:
      containers:
      - image: myapp:v2
        name: myapp
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: myapp-v2
  name: myapp-v2
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: myapp-v2

[root@k8s-master app]# kubectl expose deployment myapp-v1 --port 80 --target-port 80 --dry-run=client -o yaml >> myapp-v1.yaml

[root@k8s-master app]# kubectl expose deployment myapp-v2 --port 80 --target-port 80 --dry-run=client -o yaml >> myapp-v1.yaml


[root@k8s-master app]# kubectl get services
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP   29h
myapp-v1     ClusterIP   10.104.84.65     <none>        80/TCP    13s
myapp-v2     ClusterIP   10.105.246.219   <none>        80/TCP    7s

2.建立ingress的yaml

[root@k8s-master app]# vim ingress1.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /		#访问路径后加任何内容都被定向到/
  name: ingress1
spec:
  ingressClassName: nginx
  rules:
  - host: www.strongjcl.org
    http:
      paths:
      - backend:
          service:
            name: myapp-v1
            port:
              number: 80
        path: /v1
        pathType: Prefix

      - backend:
          service:
            name: myapp-v2
            port:
              number: 80
        path: /v2
        pathType: Prefix

#测试:
[root@reg ~]# echo 172.25.254.50 www.strongjcl.org >> /etc/hosts

[root@reg ~]# curl  www.strongjcl.org/v1
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@reg ~]# curl  www.strongjcl.org/v2
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>


#nginx.ingress.kubernetes.io/rewrite-target: / 的功能实现
[root@reg ~]# curl  www.strongjcl.org/v2/aaaa
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>

5.3.2 基于域名的访问

#在测试主机中设定解析
[root@reg ~]# vim /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
172.25.254.250 reg.strongjcl.org
172.25.254.50 www.strongjcl.org myappv1.strongjcl.org myappv2.strongjcl.org

# 建立基于域名的yml文件
[root@k8s-master app]# vim ingress2.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
  name: ingress2
spec:
  ingressClassName: nginx
  rules:
  - host: myappv1.strongjcl.org
    http:
      paths:
      - backend:
          service:
            name: myapp-v1
            port:
              number: 80
        path: /
        pathType: Prefix
  - host: myappv2.strongjcl.org
    http:
      paths:
      - backend:
          service:
            name: myapp-v2
            port:
              number: 80
        path: /
        pathType: Prefix
        
#利用文件建立ingress
[root@k8s-master app]# kubectl apply -f ingress2.yml
ingress.networking.k8s.io/ingress2 created

[root@k8s-master app]# kubectl describe ingress ingress2
Name:             ingress2
Labels:           <none>
Namespace:        default
Address:
Ingress Class:    nginx
Default backend:  <default>
Rules:
  Host                   Path  Backends
  ----                   ----  --------
  myappv1.strongjcl.org
                         /   myapp-v1:80 (10.244.2.31:80)
  myappv2.strongjcl.org
                         /   myapp-v2:80 (10.244.2.32:80)
Annotations:             nginx.ingress.kubernetes.io/rewrite-target: /
Events:
  Type    Reason  Age   From                      Message
  ----    ------  ----  ----                      -------
  Normal  Sync    21s   nginx-ingress-controller  Scheduled for sync


#在测试主机中测试
[root@reg ~]# curl  www.strongjcl.org/v1
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@reg ~]# curl  www.strongjcl.org/v2
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>

5.3.3 建立tls加密

#建立证书
[root@k8s-master app]# openssl req -newkey rsa:2048 -nodes -keyout tls.key -x509 -days 365 -subj "/CN=nginxsvc/O=nginxsvc" -out tls.crt
#建立加密资源类型secret
[root@k8s-master app]# kubectl create secret tls  web-tls-secret --key tls.key --cert tls.crt
secret/web-tls-secret created
[root@k8s-master app]# kubectl get secrets
NAME             TYPE                DATA   AGE
web-tls-secret   kubernetes.io/tls   2      6s
#建立ingress3基于tls认证的yml文件
[root@k8s-master app]# vim ingress3.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
  name: ingress3
spec:
  tls:
  - hosts:
    - myapp-tls.strongjcl.org
    secretName: web-tls-secret
  ingressClassName: nginx
  rules:
  - host: myapp-tls.strongjcl.org
    http:
      paths:
      - backend:
          service:
            name: myapp-v1
            port:
              number: 80
        path: /
        pathType: Prefix
        
#测试
[root@reg ~]# curl -k https://myapp-tls.strongjcl.org
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>

5.3.4 建立auth认证

#建立认证文件
[root@k8s-master app]# dnf install httpd-tools -y
[root@k8s-master app]# htpasswd -cm auth lee
New password:
Re-type new password:
Adding password for user lee
[root@k8s-master app]# cat auth
lee:$apr1$BohBRkkI$hZzRDfpdtNzue98bFgcU10

#建立认证类型资源
[root@k8s-master app]# kubectl create secret generic auth-web --from-file auth
root@k8s-master app]# kubectl describe secrets auth-web
Name:         auth-web
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
auth:  42 bytes
#建立ingress4基于用户认证的yaml文件
[root@k8s-master app]# vim ingress4.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/auth-type: basic
    nginx.ingress.kubernetes.io/auth-secret: auth-web
    nginx.ingress.kubernetes.io/auth-realm: "Please input username and password"
  name: ingress4
spec:
  tls:
  - hosts:
    - myapp-tls.strongjcl.org
    secretName: web-tls-secret
  ingressClassName: nginx
  rules:
  - host: myapp-tls.strongjcl.org
    http:
      paths:
      - backend:
          service:
            name: myapp-v1
            port:
              number: 80
        path: /
        pathType: Prefix

#建立ingress4
[root@k8s-master app]# kubectl apply -f ingress4.yml
ingress.networking.k8s.io/ingress4 created
[root@k8s-master app]# kubectl describe ingress ingress4
Name:             ingress4
Labels:           <none>
Namespace:        default
Address:
Ingress Class:    nginx
Default backend:  <default>
TLS:
  web-tls-secret terminates myapp-tls.strongjcl.org
Rules:
  Host                     Path  Backends
  ----                     ----  --------
  myapp-tls.strongjcl.org
                           /   myapp-v1:80 (10.244.2.31:80)
Annotations:               nginx.ingress.kubernetes.io/auth-realm: Please input username and password
                           nginx.ingress.kubernetes.io/auth-secret: auth-web
                           nginx.ingress.kubernetes.io/auth-type: basic
Events:
  Type    Reason  Age   From                      Message
  ----    ------  ----  ----                      -------
  Normal  Sync    14s   nginx-ingress-controller  Scheduled for sync


#测试:
[root@reg ~]# curl -k https://myapp-tls.strongjcl.org
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@reg ~]# vim /etc/hosts ^C
[root@reg ~]# curl -k https://myapp-tls.strongjcl.org
<html>
<head><title>401 Authorization Required</title></head>
<body>
<center><h1>401 Authorization Required</h1></center>
<hr><center>nginx</center>
</body>
</html>

[root@reg ~]# curl -k https://myapp-tls.strongjcl.org -ulee:lee
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>

5.3.5 rewrite重定向

#指定默认访问的文件到hostname.html上
[root@k8s-master app]# vim ingress5.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/app-root: /hostname.html
    nginx.ingress.kubernetes.io/auth-type: basic
    nginx.ingress.kubernetes.io/auth-secret: auth-web
    nginx.ingress.kubernetes.io/auth-realm: "Please input username and password"
  name: ingress5
spec:
  tls:
  - hosts:
    - myapp-tls.strongjcl.org
    secretName: web-tls-secret
  ingressClassName: nginx
  rules:
  - host: myapp-tls.strongjcl.org
    http:
      paths:
      - backend:
          service:
            name: myapp-v1
            port:
              number: 80
        path: /
        pathType: Prefix
[root@k8s-master app]# kubectl apply -f ingress5.yml
ingress.networking.k8s.io/ingress5 created
[root@k8s-master app]# kubectl describe ingress ingress5
Name:             ingress5
Labels:           <none>
Namespace:        default
Address:          172.25.254.10
Ingress Class:    nginx
Default backend:  <default>
TLS:
  web-tls-secret terminates myapp-tls.strongjcl.org
Rules:
  Host                     Path  Backends
  ----                     ----  --------
  myapp-tls.timinglee.org
                           /   myapp-v1:80 (10.244.2.31:80)
Annotations:               nginx.ingress.kubernetes.io/app-root: /hostname.html
                           nginx.ingress.kubernetes.io/auth-realm: Please input username and password
                           nginx.ingress.kubernetes.io/auth-secret: auth-web
                           nginx.ingress.kubernetes.io/auth-type: basic
Events:
  Type    Reason  Age                    From                      Message
  ----    ------  ----                   ----                      -------
  Normal  Sync    2m16s (x2 over 2m54s)  nginx-ingress-controller  Scheduled for sync

#测试:
[root@reg ~]# curl -Lk https://myapp-tls.strongjcl.org -ulee:lee
myapp-v1-7479d6c54d-j9xc6

[root@reg ~]# curl -Lk https://myapp-tls.strongjcl.org/lee/hostname.html -ulee:lee
<html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.12.2</center>
</body>
</html>


#解决重定向路径问题
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/auth-type: basic
    nginx.ingress.kubernetes.io/auth-secret: auth-web
    nginx.ingress.kubernetes.io/auth-realm: "Please input username and password"
  name: ingress6
spec:
  tls:
  - hosts:
    - myapp-tls.strongjcl.org
    secretName: web-tls-secret
  ingressClassName: nginx
  rules:
  - host: myapp-tls.strongjcl.org
    http:
      paths:
      - backend:
          service:
            name: myapp-v1
            port:
              number: 80
        path: /
        pathType: Prefix
      - backend:
          service:
            name: myapp-v1
            port:
              number: 80
        path: /lee(/|$)(.*)					#正则表达式匹配/lee/,/lee/abc
        pathType: ImplementationSpecific


#测试
[root@reg ~]# curl -Lk https://myapp-tls.strongjcl.org/lee/hostname.html -ulee:lee
myapp-v1-7479d6c54d-j9xc6

六 、Canary金丝雀发布

6.1 么是金丝雀发布

金丝雀发布(Canary Release)也称为灰度发布,是一种软件发布策略。

主要目的是在将新版本的软件全面推广到生产环境之前,先在一小部分用户或服务器上进行测试和验证,以降低因新版本引入重大问题而对整个系统造成的影响。

是一种Pod的发布方式。金丝雀发布采取先添加、再删除的方式,保证Pod的总量不低于期望值。并且在更新部分Pod后,暂停更新,当确认新Pod版本运行正常后再进行其他版本的Pod的更新。

6.2 Canary发布方式

其中header和weiht中的最多

6.2.1 基于header(http包头)灰度

  • 通过Annotaion扩展

  • 创建灰度ingress,配置灰度头部key以及value

  • 灰度流量验证完毕后,切换正式ingress到新版本

  • 之前我们在做升级时可以通过控制器做滚动更新,默认25%利用header可以使升级更为平滑,通过key 和vule 测试新的业务体系是否有问题。

示例:

#建立版本1的ingress
[root@k8s-master app]# vim ingress7.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
  name: myapp-v1-ingress
spec:
  ingressClassName: nginx
  rules:
  - host: myapp.strongjcl.org
    http:
      paths:
      - backend:
          service:
            name: myapp-v1
            port:
              number: 80
        path: /
        pathType: Prefix
        
[root@k8s-master app]# kubectl describe ingress myapp-v1-ingress
Name:             myapp-v1-ingress
Labels:           <none>
Namespace:        default
Address:          172.25.254.10
Ingress Class:    nginx
Default backend:  <default>
Rules:
  Host                 Path  Backends
  ----                 ----  --------
  myapp.strongjcl.org
                       /   myapp-v1:80 (10.244.2.31:80)
Annotations:           <none>
Events:
  Type    Reason  Age                From                      Message
  ----    ------  ----               ----                      -------
  Normal  Sync    44s (x2 over 73s)  nginx-ingress-controller  Scheduled for sync


#建立基于header的ingress
[root@k8s-master app]# vim ingress8.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: version
    nginx.ingress.kubernetes.io/canary-by-header-value: 2
  name: myapp-v2-ingress
spec:
  ingressClassName: nginx
  rules:
  - host: myapp.strongjcl.org
    http:
      paths:
      - backend:
          service:
            name: myapp-v2
            port:
              number: 80
        path: /
        pathType: Prefix
[root@k8s-master app]# kubectl apply -f ingress8.yml
ingress.networking.k8s.io/myapp-v2-ingress created
[root@k8s-master app]# kubectl describe ingress myapp-v2-ingress
Name:             myapp-v2-ingress
Labels:           <none>
Namespace:        default
Address:
Ingress Class:    nginx
Default backend:  <default>
Rules:
  Host                 Path  Backends
  ----                 ----  --------
  myapp.timinglee.org
                       /   myapp-v2:80 (10.244.2.32:80)
Annotations:           nginx.ingress.kubernetes.io/canary: true
                       nginx.ingress.kubernetes.io/canary-by-header: version
                       nginx.ingress.kubernetes.io/canary-by-header-value: 2
Events:
  Type    Reason  Age   From                      Message
  ----    ------  ----  ----                      -------
  Normal  Sync    21s   nginx-ingress-controller  Scheduled for sync

#测试:
[root@reg ~]# curl  myapp.strongjcl.org
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@reg ~]# curl -H "version: 2" myapp.timinglee.org
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>

6.2.2 基于权重的灰度发布

  • 通过Annotaion拓展

  • 创建灰度ingress,配置灰度权重以及总权重

  • 灰度流量验证完毕后,切换正式ingress到新版本

示例

#基于权重的灰度发布
[root@k8s-master app]# vim ingress8.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "10"		#更改权重值
    nginx.ingress.kubernetes.io/canary-weight-total: "100"
  name: myapp-v2-ingress
spec:
  ingressClassName: nginx
  rules:
  - host: myapp.strongjcl.org
    http:
      paths:
      - backend:
          service:
            name: myapp-v2
            port:
              number: 80
        path: /
        pathType: Prefix

[root@k8s-master app]# kubectl apply -f ingress8.yml
ingress.networking.k8s.io/myapp-v2-ingress created

#测试:
[root@reg ~]# vim check_ingress.sh
#!/bin/bash
v1=0
v2=0

for (( i=0; i<100; i++))
do
    response=`curl -s myapp.#基于权重的灰度发布
[root@k8s-master app]# vim ingress8.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "10"		#更改权重值
    nginx.ingress.kubernetes.io/canary-weight-total: "100"
  name: myapp-v2-ingress
spec:
  ingressClassName: nginx
  rules:
  - host: myapp.strongjcl.org
    http:
      paths:
      - backend:
          service:
            name: myapp-v2
            port:
              number: 80
        path: /
        pathType: Prefix

[root@k8s-master app]# kubectl apply -f ingress8.yml
ingress.networking.k8s.io/myapp-v2-ingress created

#测试:
[root@reg ~]# vim check_ingress.sh
#!/bin/bash
v1=0
v2=0

for (( i=0; i<100; i++))
do
    response=`curl -s myapp.strongjcl.org |grep -c v1`

    v1=`expr $v1 + $response`
    v2=`expr $v2 + 1 - $response`

done
echo "v1:$v1, v2:$v2"

[root@reg ~]# sh check_ingress.sh
v1:90, v2:10

#更改完毕权重后继续测试可观察变化.org |grep -c v1`

    v1=`expr $v1 + $response`
    v2=`expr $v2 + 1 - $response`

done
echo "v1:$v1, v2:$v2"

[root@reg ~]# sh check_ingress.sh
v1:90, v2:10

#更改完毕权重后继续测试可观察变化

  • 16
    点赞
  • 16
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值