k8s之ingress

一、基本概念
1、Kubernetes 暴露服务的方式:
NodePort：后期维护困难,不支持虚拟路径
LoadBlancer：需要云厂商支持,有局限性
ClusterIP：只能在集群内部访问
Ingress：灵活,无依赖
前三种方式都是在service的维度提供的，service的作用体现在两个方面，对集群内部它不断跟踪pod变化，更新endpoint中对应的pod的对象，提供IP不断变化的pod的服务发现机制；对集群外部，它类似负载均衡器，可以在集群内外对pod进行访问。但是单独使用service暴露服务的方式，在实际生产环境中不太适应。
而ingress相当于service的service，可以将外部请求通过不同规则的筛选后转发到不同的service

2、ingress与ingress-controller
ingress是k8s中的一个api对象，一般用yaml配置。作用是定义请求转发的规则，可以理解为配置模板或者配置文件。
ingress-controller是具体实现反向代理及负载均衡的程序，通过对ingress中定义的规则进行解析，根据其规则来实现转发。
ingress-controller并不是k8s自带的组件，实际上它只是一个统称，用户可以选择不同的ingress-controller来实现功能。其中由k8s官方维护的是nginx-ingress。

二、配置方式
1、Deployment+LoadBalancer模式的Service
如果要把ingress部署在公有云，那用这种方式比较合适。用Deployment部署ingress-controller，创建一个type为LoadBalancer的service关联这组pod。大部分公有云，都会为LoadBalancer的service自动创建一个负载均衡器，通常还绑定了公网地址。只要把域名解析指向该地址，就实现了集群服务的对外暴露

2、Deployment+NodePort模式的Service
同样用deployment模式部署ingress-controller，并创建对应的服务，但是type为NodePort。这样，ingress就会暴露在集群节点ip的特定端口上。由于nodeport暴露的端口是随机端口，一般会在前面再搭建一套负载均衡器来转发请求。该方式一般用于宿主机是相对固定的环境ip地址不变的场景。
NodePort方式暴露ingress虽然简单方便，但是NodePort多了一层NAT，在请求量级很大时可能对性能会有一定影响

3、DaemonSet+HostNetwork+nodeSelector
用DaemonSet结合nodeselector来部署ingress-controller到特定的node上（也可结合affinity亲和性部署到多个节点），然后使用HostNetwork直接把该pod与宿主机node的网络打通，直接使用宿主机的80/433端口就能访问服务。这时，ingress-controller所在的node机器就很类似传统架构的边缘节点，比如机房入口的nginx服务器。该方式整个请求链路最简单，性能相对NodePort模式更好。缺点是由于直接利用宿主机节点的网络和端口，一个node只能部署一个ingress-controller pod。比较适合大并发的生产环境使用。

三、部署实例
建立命名空间：

namespace：
apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx

1、rbac

（1）基本概念
RBAC(Role-Based Access Control，基于角色的访问控制)在k8s v1.5中引入，在v1.6版本时升级为Beta版本，并成为kubeadm安装方式下的默认选项，相对于其他访问控制方式，新的RBAC具有如下优势：
对集群中的资源和非资源权限均有完整的覆盖
整个RBAC完全由几个API对象完成，同其他API对象一样，可以用kubectl或API进行操作
可以在运行时进行调整，无需重启API Server
要使用RBAC授权模式，需要在API Server的启动参数中加上--authorization-mode=RBAC

（2）RBAC的API资源对象说明
参数说明：
apiGroup：支持的API组列表，例如：APIVersion: batch/v1、APIVersion: extensions:v1、apiVersion:apps/v1等
resources：支持的资源对象列表，例如：pods、deployments、jobs等
verbs：对资源对象的操作方法列表，例如：get、watch、list、delete、replace、patch等

角色(Role)：
一个角色就是一组权限的集合，这里的权限都是许可形式的，不存在拒绝的规则。在一个命名空间中，可以用角色来定义一个角色，如果是集群级别的，就需要使用ClusterRole了。角色只能对命名空间内的资源进行授权

集群角色(ClusterRole)：
集群角色除了具有和角色一致的命名空间内资源的管理能力，因其集群级别的范围，还可以用于以下特殊元素的授权。
集群范围的资源，例如Node
非资源型的路径，例如/healthz
包含全部命名空间的资源，例如pods

角色绑定(RoleBinding)和集群角色绑定(ClusterRoleBinding)：
角色绑定或集群角色绑定用来把一个角色绑定到一个目标上，绑定目标可以是User、Group或者Service Account。使用RoleBinding为某个命名空间授权，ClusterRoleBinding为集群范围内授权。
RoleBinding可以引用Role进行授权
RoleBinding也可以引用ClusterRole，对属于同一命名空间内ClusterRole定义的资源主体进行授权。一种常见的做法是集群管理员为集群范围预先定义好一组角色(ClusterRole)，然后在多个命名空间中重复使用这些ClusterRole。
集群角色绑定中的角色只能是集群角色，用于进行集群级别或者对所有命名空间都生效的授权。

（3）yaml文件

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
        - events
    verbs:
        - create
        - patch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses/status
    verbs:
      - update
---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      # Defaults to "<election-id>-<ingress-class>"
      # Here: "<ingress-controller-leader>-<nginx>"
      # This has to be adapted if you change either parameter
      # when launching the nginx-ingress-controller.
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

2、configmap

kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx

kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx

3、ingress-controller

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx 
spec:
#  replicas: 1
  selector:
    matchLabels:
      app: ingress-nginx
  template:
    metadata:
      labels:
        app: ingress-nginx
      annotations:
        prometheus.io/port: '10254'
        prometheus.io/scrape: 'true'
    spec:
      serviceAccountName: nginx-ingress-serviceaccount
      hostNetwork: true
      #也可使用nodeslector或不指定node
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/hostname
                operator: In
                values:
                - master
      containers:
        - name: nginx-ingress-controller
          image: registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:0.17.1
          args:
            - /nginx-ingress-controller
            - --default-backend-service=ingress-nginx/default-http-backend
            - --configmap=ingress-nginx/nginx-configuration
            - --tcp-services-configmap=ingress-nginx/tcp-services
            - --udp-services-configmap=ingress-nginx/udp-services
#            - --annotations-prefix=nginx.ingress.kubernetes.io
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
          - name: http
            containerPort: 80
          - name: https
            containerPort: 443
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1 

4、backend

apiVersion: apps/v1
kind: Deployment
metadata:
  name: default-http-backend
  labels:
    app: default-http-backend
  namespace: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: default-http-backend
  template:
    metadata:
      labels:
        app: default-http-backend
    spec:
      terminationGracePeriodSeconds: 60
      containers:
      - name: default-http-backend
        # Any image is permissable as long as:
        # 1. It serves a 404 page at /
        # 2. It serves 200 on a /healthz endpoint
        image: registry.cn-hangzhou.aliyuncs.com/google_containers/defaultbackend:1.4
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 30
          timeoutSeconds: 5
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: 10m
            memory: 20Mi
          requests:
            cpu: 10m
            memory: 20Mi
---
apiVersion: v1
kind: Service
metadata:
  name: default-http-backend
  namespace: ingress-nginx
  labels:
    app: default-http-backend
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: default-http-backend

5、部署实例及配置ingress（ingress与对应实例需在同一命名空间）

apiVersion: v1
kind: Namespace
metadata:
  name: test

---

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: testnginx
  name: test-service
  namespace: test
spec:
  ports:
    - port: 80
      targetPort: 80
  selector:
    k8s-app: testnginx

---

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: testnginx
  name: test-deployment
  namespace: test
spec:
  replicas: 3
  revisionHistoryLimit: 3
  selector:
    matchLabels:
      k8s-app: testnginx
  template:
    metadata:
      labels:
        k8s-app: testnginx
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/hostname
                operator: In
                values:
                - test1
                - test2
                - master
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            preference:
              matchExpressions:
              - key: kubernetes.io/hostname
                operator: In
                values:
                - test1
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: k8s-app
                operator: In
                values:
                - testnginx
            topologyKey: kubernetes.io/hostname
      containers:
        - name: test-pod
          image: nginx
          imagePullPolicy: Always
          ports:
            - containerPort: 80
              protocol: TCP
          volumeMounts:
          - mountPath: /usr/share/nginx/html
            name: test-volume
            subPath: html
          - mountPath: /var/log/nginx
            name: test-volume
            subPath: log
      volumes:
      - name: test-volume
        hostPath:
          path: /mnt

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-demo
  namespace: test
spec:
  rules:
  - host: test.test.test
    http:
      paths:
      - path: /
        backend:
          serviceName: test-service
          servicePort: 80