20_2.服务暴露-ingress (ingress-nginx)

20_2.服务暴露-ingress (ingress-nginx)

ingress-nginx

https://github.com/kubernetes/ingress-nginx

准备镜像

docker pull quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0

资源配置清单

参考：https://github.com/kubernetes/ingress-nginx/blob/master/deploy/static/provider/cloud/deploy.yaml
参考：https://hub.fastgit.org/kubernetes/ingress-nginx/blob/nginx-0.30.0/deploy/static/mandatory.yaml

https://hub.fastgit.org 为github的国内加速代理

namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
---

configmap.yaml

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
---

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      # wait up to five minutes for the drain of connections
      terminationGracePeriodSeconds: 300
      serviceAccountName: nginx-ingress-serviceaccount
      nodeSelector:
        kubernetes.io/os: linux
      containers:
        - name: nginx-ingress-controller
          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0
          args:
            - /nginx-ingress-controller
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            # www-data -> 101
            runAsUser: 101
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - name: http
              containerPort: 80
            - name: https
              containerPort: 443
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown
---

service.yaml

apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  type: NodePort
  ports:
    - name: http
      port: 80
      targetPort: 80
      protocol: TCP
      nodePort: 8480  # 设置对外http暴露入口为8480
    - name: https
      port: 443
      targetPort: 443
      protocol: TCP
      nodePort: 8443  # 设置对外https暴露入口为8443
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
---

使用nodePort类型service暴露ingress-nginx服务，指定端口8480和8443，作为集群服务的总入口

和traefik一样，前置代理服务器可以配置流量负载到nodeIP:8480上，后续通过ingress规则分发业务流量给目标service

rbac.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses/status
    verbs:
      - update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      # Defaults to "<election-id>-<ingress-class>"
      # Here: "<ingress-controller-leader>-<nginx>"
      # This has to be adapted if you change either parameter
      # when launching the nginx-ingress-controller.
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx
---

查看部署情况

➜ ~  kubectl get all -n ingress-nginx -o wide
NAME                                            READY   STATUS    RESTARTS   AGE     IP             NODE            NOMINATED NODE   READINESS GATES
pod/nginx-ingress-controller-69c8b48fd6-wj7kk   1/1     Running   0          3m15s   172.20.102.3   172.10.10.102   <none>           <none>
NAME                    TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)                    AGE     SELECTOR
service/ingress-nginx   NodePort   192.168.16.78   <none>        80:8480/TCP,443:8443/TCP   3m10s   app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx
NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS                 IMAGES                                                 SELECTOR
deployment.apps/nginx-ingress-controller   1/1     1            1           3m15s   nginx-ingress-controller   harbor.hzwod.com/k8s/nginx-ingress-controller:0.30.0   app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx
NAME                                                  DESIRED   CURRENT   READY   AGE     CONTAINERS                 IMAGES                                                 SELECTOR
replicaset.apps/nginx-ingress-controller-69c8b48fd6   1         1         1       3m15s   nginx-ingress-controller   harbor.hzwod.com/k8s/nginx-ingress-controller:0.30.0   app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx,pod-template-hash=69c8b48fd6

kubelctl apply上述资源后

➜  nginx-ingress curl http://172.10.10.101:8480
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.17.8</center>
</body>
</html>

看见了一个nginx？？ 其实就是nginx pod

配置ingress规则

目标服务

➜  nginx-ingress kubectl get all -n default -l name=whoami -o wide
NAME                          READY   STATUS    RESTARTS   AGE   IP             NODE            NOMINATED NODE   READINESS GATES
pod/whoami-644f4f96df-f7dcj   1/1     Running   0          12m   172.20.102.3   172.10.10.102   <none>           <none>
NAME             TYPE        CLUSTER-IP        EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/whoami   ClusterIP   192.168.255.128   <none>        80/TCP    17m   app=myapp,task=whoami

ingress规则

whoami_ingress.yaml

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: whoamiingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - http:
      paths:
        - path: /bar
          backend:
            serviceName: "whoami"
            servicePort: 80

测试

➜  nginx-ingress kubectl get ingress --all-namespaces
NAMESPACE   NAME            HOSTS   ADDRESS           PORTS   AGE
default     whoamiingress   *       192.168.175.255   80      12m
➜  nginx-ingress curl http://172.10.10.101:8480/bar
Hostname: whoami-644f4f96df-f7dcj
IP: 127.0.0.1
IP: 172.20.102.3
RemoteAddr: 172.20.102.4:45984
GET /bar HTTP/1.1
Host: 172.10.10.101:8480
User-Agent: curl/7.29.0
Accept: */*
X-Forwarded-For: 172.10.10.101
X-Forwarded-Host: 172.10.10.101:8480
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Real-Ip: 172.10.10.101
X-Request-Id: 34cbd03ec296a29f6e4918bc2236906e
X-Scheme: http

实现了通过ingress-nginx入口代理访问到了集群内部whoami的服务