k8s中通过ingress暴露多端口deployment

前言

项目中有一个notify微服务，业务逻辑上，需要在web界面上操作发送模板（微服务）、和推送（websocket），因此需要将后端的微服务和websocket同时对外暴露，前端web界面操作时需要走外网，同时实现微服务内部之间和notify访问，仍然走内网。

对外暴露配置

deployment.yml

如下面的代码所示，ports部分有微服务8190和websocket的9050两个targetport。

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: '17'
    meta.helm.sh/release-name: notify
    meta.helm.sh/release-namespace: sc-dev
  labels:
    app.kubernetes.io/instance: notify
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: fr-sc
    app.kubernetes.io/version: 1.16.0
    helm.sh/chart: fr-sc-0.1.0
  name: fr-sc-notify
  namespace: sc-dev
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/instance: notify
      app.kubernetes.io/name: fr-sc
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        app.kubernetes.io/instance: notify
        app.kubernetes.io/name: fr-sc
    spec:
      containers:
        - image: 'harbor.XXXX.com/sc/fr-sc-notify:77'
          imagePullPolicy: IfNotPresent
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /actuator/health
              port: 8190
              scheme: HTTP
            initialDelaySeconds: 5
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 5
          name: fr-sc-notify
          ports:
            - containerPort: 8190
              name: http
              protocol: TCP
            - containerPort: 9050
              name: websocket
              protocol: TCP
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /actuator/health
              port: 8190
              scheme: HTTP
            initialDelaySeconds: 5
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 5
      imagePullSecrets:
        - name: registry-pull-secret
      restartPolicy: Always

service.yml

service中各自用相同端口将deployment中的服务暴露出来，注意websocket传输层上仍然处于tcp协议。

apiVersion: v1
kind: Service
metadata:
  name: fr-sc-notify
  namespace: sc-dev
spec:
  ports:
    - name: notify-request
      port: 8190
      protocol: TCP
      targetPort: 8190
    - name: notify-ws
      port: 9050
      protocol: TCP
      targetPort: 9050
  selector:
    app.kubernetes.io/instance: notify
    app.kubernetes.io/name: fr-sc
  type: ClusterIP

ingress.yml

注意这里有个小技巧，微服务后端，ws访问方式配置成：uri.contains("/websocket") ，微服务的请求方式配置成：@RequestMapping("/ws_publish")，则下面ingress的path部分与上述保持一致，如下所示，这样访问ws时，可以直接ws://notify.abc.com/websocket/XX/XX的形式访问，同理，微服务可以通过http://notify.abc.com/ws_publish/XX/XX的形式访问。

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: fr-ingressclass
  labels:
    app.kubernetes.io/instance: notify
    app.kubernetes.io/name: fr-sc
  name: fr-sc-notify
  namespace: sc-dev
spec:
  rules:
    - host: notify.abc.com
      http:
        paths:
          - backend:
              service:
                name: fr-sc-notify
                port:
                  number: 8190
            path: /ws_publish
            pathType: ImplementationSpecific
          - backend:
              service:
                name: fr-sc-notify
                port:
                  number: 9050
            path: /websocket
            pathType: ImplementationSpecific

对内暴露

当微服务内部之间需要互访，就没有必要通过公网的方式了，本文通过阿里ack配置的，上述配置一个对外的elb，此处再创建一个intranet SLB，用阿里云解析PrivateZone配置解析即可。关于slb的创建配置，可以参考本人此篇文章。