Kubernetes - 创建Ingress

kubernetes 1.6.2 搭建ingress。Ingress 用来代理后端服务，它就是k8s下的nginx。

安装Ingress

• Ingress需要一个默认的后端，创建default-backend.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: default-http-backend
  labels:
    k8s-app: default-http-backend
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        k8s-app: default-http-backend
    spec:
      terminationGracePeriodSeconds: 60
      containers:
      - name: default-http-backend
        # Any image is permissable as long as:
        # 1. It serves a 404 page at /
        # 2. It serves 200 on a /healthz endpoint
        image: gcr.io/google_containers/defaultbackend:1.0
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 30
          timeoutSeconds: 5
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: 10m
            memory: 20Mi
          requests:
            cpu: 10m
            memory: 20Mi
---
apiVersion: v1
kind: Service
metadata:
  name: default-http-backend
  namespace: kube-system
  labels:
    k8s-app: default-http-backend
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    k8s-app: default-http-backend

• 在k8s 1.6.2下需要创建Role、ClusterRole、RoleBinding、ClusterRoleBinding，创建ingress-role.yaml

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: clusterrole-ingress
rules:
- apiGroups:
  - ""
  - "extensions"
  resources:
  - configmaps
  - secrets
  - services
  - endpoints
  - ingresses
  - nodes
  - pods
  verbs:
  - list
  - watch
- apiGroups:
  - "extensions"
  resources:
  - ingresses
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - events
  - services
  verbs:
  - create
  - list
  - update
  - get
- apiGroups:
  - "extensions"
  resources:
  - ingresses/status
  - ingresses
  verbs:
  - update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: role-ingress
  namespace: kube-system
rules:
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - list
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - endpoints
  verbs:
  - get
  - create
  - update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: ingress-clusterrolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: clusterrole-ingress
subjects:
  - kind: ServiceAccount
    name: ingress
    namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: ingress-rolebinding
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: role-ingress
subjects:
  - kind: ServiceAccount
    name: ingress
    namespace: kube-system

• 创建服务账号ingress-ServiceAccount.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: ingress
  namespace: kube-system

• 创建controller，nginx-ingress-controller.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  labels:
    k8s-app: nginx-ingress-controller
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        k8s-app: nginx-ingress-controller
      annotations:
        prometheus.io/port: '10254'
        prometheus.io/scrape: 'true'
    spec:
      # hostNetwork makes it possible to use ipv6 and to preserve the source IP correctly regardless of docker configuration
      # however, it is not a hard dependency of the nginx-ingress-controller itself and it may cause issues if port 10254 already is taken on the host
      # that said, since hostPort is broken on CNI (https://github.com/kubernetes/kubernetes/issues/31307) we have to use hostNetwork where CNI is used
      # like with kubeadm
      # hostNetwork: true
      terminationGracePeriodSeconds: 60
      serviceAccountName: ingress
      containers:
      - image: gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.5
        name: nginx-ingress-controller
        readinessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
        livenessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          timeoutSeconds: 1
        ports:
        - containerPort: 80
          hostPort: 80
        - containerPort: 443
          hostPort: 443
        env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
        args:
        - /nginx-ingress-controller
        - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
      nodeSelector:
        kubernetes.io/hostname: 192.168.1.211

这里通过nodeSelector绑定了pod启动的节点，将外部请求转发到这个节点上。
查看node label的方法：

[root@k8s-master ingress]# kubectl describe node 192.168.1.211
Name:           192.168.1.211
Role:
Labels:         beta.kubernetes.io/arch=amd64
            beta.kubernetes.io/os=linux
            kubernetes.io/hostname=192.168.1.211
...

通过serviceAccountName，指定了pod使用的服务账号，也就是将上面创建的角色绑定起来。

至此ingress搭建完毕

---

测试ingress

创建服务frontend （《Kubernetes权威指南》中的例子）。

创建frontend-controller.yaml

apiVersion: v1
kind: ReplicationController
metadata:
  name: frontend
  labels:
    name: frontend
spec:
  replicas: 1
  selector:
    name: frontend
  template:
    metadata:
      labels:
        name: frontend
    spec:
      containers:
      - name: frontend
        image: kubeguide/guestbook-php-frontend:latest
        env:
        - name: GET_HOSTS_FROM
          value: env
        ports:
        - containerPort: 80

创建frontend-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: frontend
  labels:
    name: frontend
spec:
  type: NodePort
  ports:
  - port: 80
    targetPort: 80
  selector:
    name: frontend

查看service：

[root@k8s-master ~]# kubectl get svc
NAME         CLUSTER-IP       EXTERNAL-IP   PORT(S)       AGE
frontend     10.254.179.200   <nodes>       80:8821/TCP   1h

---

创建转发规则frontend-ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: frontend-ingress
spec:
  rules:
  - host: guestbook.test.com
    http:
      paths:
      - path: /
        backend:
          serviceName: frontend
          servicePort: 80

注意这里要写80而不是8821。否则访问出现503错误。

查看ingress

[root@k8s-master ingress]# kubectl get ing
NAME                HOSTS                  ADDRESS   PORTS     AGE
frontend-ingress    guestbook.test.com             80        1h

现在从公网将guestbook.test.com转发到ingress所在node，即可看到guestbook页面。

进入nginx-ingress查看nginx.conf

[root@k8s-master ingress]# kubectl get pods -n=kube-system | grep ingress
nginx-ingress-controller-1894093054-835nx   1/1       Running   0          3h
[root@k8s-master ingress]# kubectl exec -it nginx-ingress-controller-1894093054-835nx bash -n=kube-system
root@nginx-ingress-controller-1894093054-835nx:/# cat /etc/nginx/nginx.conf

    ...
    upstream default-frontend-80 {
        least_conn;
        server 172.30.29.7:80 max_fails=0 fail_timeout=0;
    }
    ...
    server {
        server_name guestbook.test.com;
        listen 80;
        listen [::]:80;

        location / {
        ...
        }
    }
    ...

---

更新ingress

改变frontend的pod数量，将frontend-controller.yaml中replicas由1改成3，并应用frontend-controller.yaml，重新应用frontend-ingress.yaml

[root@k8s-master frontend]#  kubectl apply -f frontend-controller.yaml
replicationcontroller "frontend" configured
[root@k8s-master frontend]# kubectl get pods
NAME                       READY     STATUS    RESTARTS   AGE
frontend-40c70             1/1       Running   0          7m
frontend-4m67t             1/1       Running   0          53m
frontend-xv1ck             1/1       Running   0          36s
[root@k8s-master ingress]# kubectl apply -f frontend-ingress.yaml
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
ingress "frontend-ingress" configured

查看ingress中nginx.conf的变化：

    upstream default-frontend-80 {
        least_conn;
        server 172.30.29.7:80 max_fails=0 fail_timeout=0;
        server 172.30.95.5:80 max_fails=0 fail_timeout=0;
        server 172.30.95.6:80 max_fails=0 fail_timeout=0;
    }

---

参考

https://github.com/kubernetes/ingress/tree/master/examples/deployment/nginx