Ingress企业实战：部署多个Ingress控制器篇

背景

在大规模集群场景中，部分服务需要通过公网Ingress对外提供服务访问，但是有部分服务只对内提供服务，不允许使用公网访问，仅支持内部服务间调用，此时可以通过部署两套独立的Ingress来实现，一套支持公网访问，一套仅支持内网访问。接下来，我们通过最佳实践进行实现喽！

架构图

最佳实践说明：

• 在Kubernetes集群中，部署两个Ingress控制器分别为int-ingress-nginx和ingress-nginx
• int-ingress-nginx负责承接内部流量
• ingress-nginx负责承接外部流量

Ingress 配置参数说明

• controller.image.repository：ingress-nginx镜像地址
• controller.image.registry：镜像仓库域名，该参数需要与controller.image.image同时填写
• controller.image.image：镜像名称。该参数需要与controller.image.registry同时填写
• controller.image.tag：ingress-nginx镜像版本，可通过已安装插件实例的YAML文件查看，需要根据插件版本进行替换
• controller.ingressClass：设置Ingress Controller所对应的IngressClass的名称，同一个集群中不同套Ingress Controller名称必须唯一
• controller.ingressClassResource.name：需要与ingressClass值相同
• controller.ingressClassResource.controllerValue：同一个集群中不同套Ingress Controller的监听标识必须唯一，且不能设置为k8s.io/ingress-nginx，（k8s.io/ingress-nginx是默认Nginx Ingress Controller的监听标识）。
• controller.resources.requests.cpu：Nginx controller的CPU资源申请值，可根据需求自定义
• controller.resources.requests.memory：Nginx controller的内存资源申请值，可根据需求自定义
• defaultBackend.image.tag：default-backend镜像版本
• controller.admissionWebhooks.enabled：是否开启admissionWebhooks，可以对Ingress对象进行有效性校验，避免因配置错误导致ingress-controller不断重新加载资源，导致业务中断。
• controller.admissionWebhooks.patch.enabled：同上，表示是否开启admissionWebhooks。此处设置为false。

Ingress控制器CRD的关联关系：

说明：

• Ingress-controller启动的时候指定controller-class名称
• 通过IngressClass CRD对象进行声明并与Ingress-controller进行绑定
• Ingress CRD对象关联IngressClassName

部署多个Ingress Nginx

本案例通过helm进行安装:

# 配置helm仓库
$ helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
"ingress-nginx" has been added to your repositories
$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "higress.io" chart repository
...Successfully got an update from the "ingress-nginx" chart repository
Update Complete. ⎈Happy Helming!⎈

安装Ingress-nginx（外网流量）：

$ helm -n ingress-nginx install ingress-nginx ingress-nginx/ingress-nginx --create-namespace --set controller.ingressClass=ingress-nginx,controller.ingressClassResource.name=ingress-nginx
NAME: ingress-nginx
LAST DEPLOYED: Tue Aug 22 14:26:25 2023
NAMESPACE: ingress-nginx
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The ingress-nginx controller has been installed.
It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status by running 'kubectl --namespace ingress-nginx get services -o wide -w ingress-nginx-controller'

An example Ingress that makes use of the controller:
  apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    name: example
    namespace: foo
  spec:
    ingressClassName: ingress-nginx
    rules:
      - host: www.example.com
        http:
          paths:
            - pathType: Prefix
              backend:
                service:
                  name: exampleService
                  port:
                    number: 80
              path: /
    # This section is only required if TLS is to be enabled for the Ingress
    tls:
      - hosts:
        - www.example.com
        secretName: example-tls

If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:

  apiVersion: v1
  kind: Secret
  metadata:
    name: example-tls
    namespace: foo
  data:
    tls.crt: <base64 encoded cert>
    tls.key: <base64 encoded key>
  type: kubernetes.io/tls

安装int-ingress-nginx（内部流量）：

$ helm -n int-ingress-nginx install int-ingress-nginx ingress-nginx/ingress-nginx --create-namespace --set controller.ingressClass=int-ingress-nginx,controller.ingressClassResource.name=int-ingress-nginx
NAME: int-ingress-nginx
LAST DEPLOYED: Tue Aug 22 14:29:00 2023
NAMESPACE: int-ingress-nginx
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The ingress-nginx controller has been installed.
It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status by running 'kubectl --namespace int-ingress-nginx get services -o wide -w int-ingress-nginx-controller'

An example Ingress that makes use of the controller:
  apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    name: example
    namespace: foo
  spec:
    ingressClassName: int-ingress-nginx
    rules:
      - host: www.example.com
        http:
          paths:
            - pathType: Prefix
              backend:
                service:
                  name: exampleService
                  port:
                    number: 80
              path: /
    # This section is only required if TLS is to be enabled for the Ingress
    tls:
      - hosts:
        - www.example.com
        secretName: example-tls

If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:

  apiVersion: v1
  kind: Secret
  metadata:
    name: example-tls
    namespace: foo
  data:
    tls.crt: <base64 encoded cert>
    tls.key: <base64 encoded key>
  type: kubernetes.io/tls

验证：

$ kubectl get pods -A
NAMESPACE            NAME                                           READY   STATUS    RESTARTS   AGE
ingress-nginx        ingress-nginx-controller-5db5867dcf-v9pfv      1/1     Running   0          3m
int-ingress-nginx    int-ingress-nginx-controller-54585986c-pp9v4   1/1     Running   0          26s
kube-system          coredns-5d78c9869d-5m4t5                       1/1     Running   0          4d22h
kube-system          coredns-5d78c9869d-bs7s4                       1/1     Running   0          4d22h
kube-system          etcd-kind-control-plane                        1/1     Running   0          4d22h
kube-system          kindnet-5cxpf                                  1/1     Running   0          4d22h
kube-system          kube-apiserver-kind-control-plane              1/1     Running   0          4d22h
kube-system          kube-controller-manager-kind-control-plane     1/1     Running   0          4d22h
kube-system          kube-proxy-c9ztm                               1/1     Running   0          4d22h
kube-system          kube-scheduler-kind-control-plane              1/1     Running   0          4d22h
local-path-storage   local-path-provisioner-6bc4bddd6b-7v8fm        1/1     Running   0          4d22h
$ kubectl get ingressClass
NAME                CONTROLLER             PARAMETERS   AGE
ingress-nginx       k8s.io/ingress-nginx   <none>       3m57s
int-ingress-nginx   k8s.io/ingress-nginx   <none>       83s

到此为止，多个ingress安装成功，接下来，咱们可以安装demo进行验证了！

部署示例服务

• Auth服务：负责用户登陆授权返回Token
• Demo服务：负责认证通过后访问的微服务

# 部署Auth服务
$ cat auth.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: auth
  labels:
    app: auth
spec: