k8s ingress 理解及配置

 

Ingress: 入口，负责统一管理外部对k8s cluster中service的请求。通过定义Ingress Resoucre（Ingress，Ingress Controller，Ingress Service）等实现。

    internet
        |
   [ Ingress ]
   --|-----|--
   [ Services ]

使用Ingress的好处：

1. 端口管理。减少不必要端口暴露，便于管理。

所有的请求，通过Ingress对应的IP：PORT进入，过滤/转发/负载均衡到相应的service/pod。

k8s现在提供三种暴露服务的方式：LoadBlancer、NodePort 、Ingress。

NodePort会在每个node上暴露对应的port，不便管理。

 

2. 动态配置服务

以Ingress Nignx为例，实现原理如下。

Ingress Contronler 通过与 Kubernetes API 交互，能够动态的获取cluster中Ingress rules的变化，生成一段 Nginx 配置，再写到 Nginx-ingress-control的 Pod 里，reload pod 使规则生效。从而实现注册的service及其对应域名/IP/Port的动态添加和解析。

 

Demo：

1. 配置ingress

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-nginx
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: java-application.local
    http:
      paths:
      - path: /spring-boot-demo
        backend:
          serviceName: spring-boot-demo
          servicePort: 8080

（注：java-application.local是配置的指向k8s所在IP的域名，eg: minikube环境下，指向minikube ip, 访问 java-application.local：30001/spring-boot-demo 即可访问 spring-boot-demo service）

2. 配置ingress controller及configmap（ingress nginx通过读取configmap 和annotations中的配置来配置nginx

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: spring-boot-demo
  labels:
    app: ingress-nginx
---

kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: spring-boot-demo
---

kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: spring-boot-demo
---


apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: spring-boot-demo
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
      app: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      serviceAccountName: nginx-ingress-serviceaccount
      containers:
      - name: nginx-ingress-controller
        image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.23.0
        args:
        - /nginx-ingress-controller
        - --configmap=$(POD_NAMESPACE)/nginx-configuration
        - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
        - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
        - --publish-service=$(POD_NAMESPACE)/ingress-nginx
        - --annotations-prefix=nginx.ingress.kubernetes.io
        securityContext:
          allowPrivilegeEscalation: true
          capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
          runAsUser: 33
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        ports:
        - name: http
          containerPort: 80
        - name: https
          containerPort: 443
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 10
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 10

---

3. 配置相关权限，从而使ingress resouce可以访问API Server，获取相关cluster/service/pod信息

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: spring-boot-demo
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  - endpoints
  - nodes
  - pods
  - secrets
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - "extensions"
  resources:
  - ingresses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - "extensions"
  resources:
  - ingresses/status
  verbs:
  - update

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: spring-boot-demo
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  - pods
  - secrets
  - namespaces
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - configmaps
  resourceNames:
  # Defaults to "<election-id>-<ingress-class>"
  # Here: "<ingress-controller-leader>-<nginx>"
  # This has to be adapted if you change either parameter
  # when launching the nginx-ingress-controller.
  - "ingress-controller-leader-nginx"
  verbs:
  - get
  - update
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - create
- apiGroups:
  - ""
  resources:
  - endpoints
  verbs:
  - get

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: spring-boot-demo
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
- kind: ServiceAccount
  name: nginx-ingress-serviceaccount
  namespace: spring-boot-demo

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
- kind: ServiceAccount
  name: nginx-ingress-serviceaccount
  namespace: spring-boot-demo

---

 4. 配置ingress service

apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: spring-boot-demo
spec:
  type: NodePort
  ports:
  - name: http
    port: 80
    targetPort: 80
    protocol: TCP
    nodePort: 30001
  selector:
    app: ingress-nginx

 

Refs:

https://kubernetes.io/docs/concepts/services-networking/ingress/

https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/

https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.13/#podtemplatelist-v1-core

https://kubernetes.github.io/ingress-nginx/deploy/

https://github.com/kubernetes/ingress-nginx/blob/master/deploy/configmap.yaml

https://www.cnblogs.com/ericnie/p/6965091.html

（Traefik针对k8s有更多预定义集成配置，需要自己配置的内容大大减少）

https://docs.traefik.io/user-guide/kubernetes/