k8s部署ingress-controller实战，以及path前缀配置的问题

最新推荐文章于 2024-10-10 10:25:03 发布

qq_37437704

最新推荐文章于 2024-10-10 10:25:03 发布

阅读量5.3k

点赞数 3

文章标签：负载均衡服务器 java

本文链接：https://blog.csdn.net/qq_37437704/article/details/121898754

版权

ingress与ingress-controller

ingress与ingress-controller的相关简介就不多介绍了，本文只写实战环节

docker镜像准备

镜像下载

docker pull registry.cn-beijing.aliyuncs.com/google_registry/nginx-ingress-controller:0.30.0

有私有仓库的，可以重命名push到私有仓库。

部署ingress-controller pod及相关资源

给节点打上标签，告诉k8那个节点可以创建ingress-controller的pod，node-1是节点名称，我们给node-1打上isIngress="true"的标签，后续我们会使用到，亦可自行再yaml内指定具体的节点

kubectl  label   node  node-1 isIngress="true"

下载两个yaml文件

https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml
https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/service-nodeport.yaml

下载不了可直接使用下面内容

mandatory.yaml：

apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses/status
    verbs:
      - update

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      # Defaults to "<election-id>-<ingress-class>"
      # Here: "<ingress-controller-leader>-<nginx>"
      # This has to be adapted if you change either parameter
      # when launching the nginx-ingress-controller.
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      # wait up to five minutes for the drain of connections
      terminationGracePeriodSeconds: 300
      serviceAccountName: nginx-ingress-serviceaccount
      nodeSelector:
        isIngress: "true"
      containers:
        - name: nginx-ingress-controller
          image: registry.cn-beijing.aliyuncs.com/google_registry/nginx-ingress-controller:0.30.0
          args:
            - /nginx-ingress-controller
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            # www-data -> 101
            runAsUser: 101
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown

---

apiVersion: v1
kind: LimitRange
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  limits:
  - min:
      memory: 90Mi
      cpu: 100m
    type: Container

文件内包括创建namespace、serviceAccount、serviceAccount权限、ingress-controller的deployment等等。

ingress-nginx服务，ingress对外暴露并接收请求的统一入口，service-nodeport.yaml：

apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  type: NodePort
  ports:
    - name: http
      port: 80
      targetPort: 80
      protocol: TCP
    - name: https
      port: 443
      targetPort: 443
      protocol: TCP
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---

运行上面两个文件

kubectl apply -f mandatory.yaml

kubectl apply -f service-nodeport.yaml

查看对应namespace下的pod与service，至此，我们已经创建好ingress-controller了，记住这两个对外暴露的端口，我们后续有用，一个是http的一个是https的。

创建ingress，配置前缀匹配规则

以tensorboard为例子，创建了两个pod与service，service名称/端口分别是tensorboard-task39-service/20021，tensorboard-task40-service/20020，创建代码就省略了。

创建两个ingress文件，tensorboard-ingress-test1.yaml，tensorboard-ingress-test2.yaml。

tensorboard-ingress-test1.yaml配置指向tensorboard-task39-service/20021

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tensorboard-ingress-test1
spec:
  ingressClassName: nginx
  rules:
  - host: tensorboard-ingress.org
    http:
      paths:
      - path: /tensorboard/39
        pathType: Prefix
        backend:
          service:
            name: tensorboard-task39-service
            port: 
              number: 20021

tensorboard-ingress-test2.yaml配置指向tensorboard-task40-service/20020

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tensorboard-ingress-test2
spec:
  ingressClassName: nginx
  rules:
  - host: tensorboard-ingress.org
    http:
      paths:
      - path: /tensorboard/40
        pathType: Prefix
        backend:
          service:
            name: tensorboard-task40-service
            port: 
              number: 20020

两份配置同样使用自己创建的域名tensorboard-ingress.org，

访问

访问前需要修改本地host，自建域名并不能被正常解析，把k8集群ip与域名配置打hosts文件内即可。

分别访问，可以得到不同的显示，这里就不展示了。

tensorboard-ingress.org:31271/tensorboard/39/

tensorboard-ingress.org:31271/tensorboard/40/

总结分析

1、生成ingress并访问，需要使用域名:ingress-nginx端口才能访问，即我们创建ingress-controller步骤所生成的service的对外暴露端口，生成多个ingress使用多个域名，访问URL也仅仅是域名改变，端口还是那个端口。

2、多个ingress可使用同一个域名，但是path部分配置，每个ingress配置需要配置不同的path规则区分需要访问的service和pod。

3、对应path的配置，想要使用同一个域名，不同path跳到不同的pod，需要pod内的应用或接口支持，例如tensorboard-ingress.org:31271/tensorboard/39/，实际上在tensorboard-task39-service所对应的pod内请求localhost:port/tensorboard/39/，ingress并不会帮你解析/tensorboard/39/这个path，只会原样的替你转发到对应的pod内请求。这个是很多文章都没有说的，只教你如何配置不同前缀去访问不同的pod。