使用DaemonSet+Taint/Tolerations+NodeSelector部署Nginx Ingress Controller

• 在Kuberntes Cluster中准备N个节点，我们称之为代理节点。在这N个节点上只部署Nginx Ingress Controller（简称NIC）实例，不会跑其他业务容器。

• 给代理节点打上NoExecute Taint，防止业务容器调度或运行在这些节点。

  
  # 给代理节点打上NoExecute Taint
  
  $kubectl taint nodes 192.168.56.105 LB=NIC:NoExecute

• 给代理节点打上Label，让NIC只部署在打了对应Lable的节点上。

  
  # 给代理节点打上对应Label
  
  $kubectl label nodes 192.168.56.105 LB=NIC

• 定义DaemonSet Yaml文件，注意加上Tolerations和Node Selector。

  apiVersion: extensions/v1beta1
  kind: DaemonSet
  metadata:
    name: nginx-ingress-lb
    labels:
      name: nginx-ingress-lb
    namespace: kube-system
  spec:
    template:
      metadata:
        labels:
          name: nginx-ingress-lb
        annotations:
          prometheus.io/port: '10254'
          prometheus.io/scrape: 'true'
      spec:
        terminationGracePeriodSeconds: 60
        # 加上对应的Node Selector
        nodeSelector:
          LB: NIC
        # 加上对应的Tolerations
        tolerations:
        - key: "LB"
          operator: "Equal"
          value: "NIC"
          effect: "NoSchedule"
        containers:
        - image: gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.11
          name: nginx-ingress-lb
          readinessProbe:
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
          livenessProbe:
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 20
            timeoutSeconds: 5
          ports:
          - containerPort: 80
            hostPort: 80
          - containerPort: 443
            hostPort: 443
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          args:
          - /nginx-ingress-controller
          - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
          - --apiserver-host=http://192.168.56.119:8090  # 这个参数很重要，否则NIC会通过kubernetes discovery去找apiserver，导致连接不上apiserver，NIC会一直重启。

• 创建default backend服务, 服务404.

  准备default-backend.yaml。

  apiVersion: extensions/v1beta1
  kind: Deployment
  metadata:
    name: default-http-backend
    labels:
      k8s-app: default-http-backend
    namespace: kube-system
  spec:
    replicas: 1
    template:
      metadata:
        labels:
          k8s-app: default-http-backend
      spec:
        terminationGracePeriodSeconds: 60
        containers:
        - name: default-http-backend
          # Any image is permissable as long as:
          # 1. It serves a 404 page at /
          # 2. It serves 200 on a /healthz endpoint
          image: gcr.io/google_containers/defaultbackend:1.0
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            timeoutSeconds: 5
          ports:
          - containerPort: 8080
          resources:
            limits:
              cpu: 10m
              memory: 20Mi
            requests:
              cpu: 10m
  
              memory: 20Mi
  ---
  
  apiVersion: v1
  kind: Service
  metadata:
    name: default-http-backend
    namespace: kube-system
    labels:
      k8s-app: default-http-backend
  spec:
    ports:
    - port: 80
      targetPort: 8080
    selector:
      k8s-app: default-http-backend
  

  根据default-backend.yaml创建对应的Deployment和Service。

  $ kubectl create -f examples/deployment/nginx/default-backend.yaml

• 根据DaemonSet Yaml创建NIC DaemonSet，启动NIC。

  $ kubectl apply -f nginx-ingress-daemonset.yaml 

• 确认NIC启动成功后，创建测试用的服务。

  $ kubectl run echoheaders --image=gcr.io/google_containers/echoserver:1.8 --replicas=1 --port=8080
  
  $ kubectl expose deployment echoheaders --port=80 --target-port=8080 --name=echoheaders-x
  
  $ kubectl expose deployment echoheaders --port=80 --target-port=8080 --name=echoheaders-y

• 创建测试用的Ingress Object。

  定义一下文件：ingress.yaml。

  apiVersion: extensions/v1beta1
  kind: Ingress
  metadata:
    name: echomap
    namespace: default
  spec:
    rules:
    - host: foo.bar.com
      http:
        paths:
        - backend:
            serviceName: echoheaders-x
            servicePort: 80
          path: /foo
    - host: bar.baz.com
      http:
        paths:
        - backend:
            serviceName: echoheaders-y
            servicePort: 80
          path: /bar
        - backend:
            serviceName: echoheaders-x
            servicePort: 80
          path: /foo

  根据ingress.yaml创建对应的Ingress。

  $ kubectl apply -f ingress.yaml

• 查看ingress的代理地址

  [root@master01 nginx]# kubectl describe ing echomap
  Name:           echomap
  Namespace:      default
  Address:        192.168.56.105 #代理地址
  Default backend:    default-http-backend: