Kong(k8s)部署及使用

一、Kong介绍

  Kong是一个开源的微服务网关，具有登录鉴权、熔断限流等功能，且其是基于Nginx和OpenResty开发的。

  同时，Kong也提供了和k8s高度融合的Kong Ingress Controller，可以直接作为k8s的原生网关使用。

  Kong从1.1版本之后开始支持dbless模式，即不需要数据库支持，可通过yaml文件进行声明式配制。postgres模式可采用Restful API进行操作，可根据以下条件选择：

  如果需要经常性的添加大量的consumer，可以选择postgres模式。

二、部署postgres

  kong和konga都需要用到数据库，所以需要有两个数据库，分别命名kong和konga。

apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
  namespace: kong
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - env:
        - name: POSTGRES_USER
          value: postgres
        - name: POSTGRES_PASSWORD
          value: postgres
        - name: POSTGRES_DB
          value: postgres
        - name: PGDATA
          value: /var/lib/postgresql/data/pgdata
        image: postgres:10
        name: postgres
        ports:
        - containerPort: 5432
        volumeMounts:
        - mountPath: /var/lib/postgresql/data
          name: postgre-data
      terminationGracePeriodSeconds: 60
      volumes:
        - name: postgre-data
          hostPath:
            path: /var/lib/postgresql/data

---
apiVersion: v1
kind: Service
metadata:
  namespace: kong
  name: postgres-service
  labels:
    app: postgres
spec:
  type: NodePort
  ports:
  - port: 5432
    targetPort: 5432
    protocol: TCP
  selector:
   app: postgres

  进入Pod，创建两个数据库。

1.进入pod
$ kubectl exec pod名 -n kong -it -- /bin/bash

2.登录数据库
$ psql -h 127.0.0.1 -p 5432 -d postgres -U postgres

3.创建kong数据库
//kong数据库
$ create user kong with password '123456';      #创建用户
$ create database kong owner kong;      #创建数据库
$ grant all privileges on database kong to kong;     #授权

//创建konga数据库
$ create user konga with password '123456';      #创建用户
$ create database konga owner konga;      #创建数据库
$ grant all privileges on database konga to konga;     #授权

三、部署Kong

1.迁移Kong数据库

  镜像就使用kong的安装镜像即可以。

apiVersion: batch/v1
kind: Job
metadata:
  name: kong-migrations
  namespace: kong
spec:
  template:
    metadata:
      name: kong-migrations
    spec:
      containers:
      - command:
        - /bin/sh
        - -c
        - kong migrations bootstrap
        env:
        - name: KONG_DATABASE
          value: postgres
        - name: KONG_PG_HOST
          value: postgres-service.kong.svc
        - name: KONG_PG_USER
          value: kong
        - name: KONG_PG_PASSWORD
          value: "123456"
        - name: KONG_PG_DATABASE
          value: kong
        - name: KONG_PG_PORT
          value: "5432"
        image: kong:2.1
        name: kong-migrations
      initContainers:
      - command:
        - /bin/sh
        - -c
        - until nc -zv $KONG_PG_HOST $KONG_PG_PORT -w1; do echo 'waiting for db'; sleep 1; done
        env:
        - name: KONG_DATABASE
          value: postgres
        - name: KONG_PG_HOST
          value: postgres-service.kong.svc
        - name: KONG_PG_USER
          value: kong
        - name: KONG_PG_PASSWORD
          value: "123456"
        - name: KONG_PG_DATABASE
          value: kong
        - name: KONG_PG_PORT
          value: "5432"
        image: busybox
        name: wait-for-postgres
      restartPolicy: OnFailure

2. 部署Kong

  注意127.0.0.1监听端口改为0.0.0.0

apiVersion: v1
kind: Namespace
metadata:
  name: kong
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: kongclusterplugins.configuration.konghq.com
spec:
  additionalPrinterColumns:
  - JSONPath: .plugin
    description: Name of the plugin
    name: Plugin-Type
    type: string
  - JSONPath: .metadata.creationTimestamp
    description: Age
    name: Age
    type: date
  - JSONPath: .disabled
    description: Indicates if the plugin is disabled
    name: Disabled
    priority: 1
    type: boolean
  - JSONPath: .config
    description: Configuration of the plugin
    name: Config
    priority: 1
    type: string
  group: configuration.konghq.com
  names:
    kind: KongClusterPlugin
    plural: kongclusterplugins
    shortNames:
    - kcp
  scope: Cluster
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      properties:
        config:
          type: object
        configFrom:
          properties:
            secretKeyRef:
              properties:
                key:
                  type: string
                name:
                  type: string
                namespace:
                  type: string
              required:
              - name
              - namespace
              - key
              type: object
          type: object
        disabled:
          type: boolean
        plugin:
          type: string
        protocols:
          items:
            enum:
            - http
            - https
            - grpc
            - grpcs
            - tcp
            - tls
            type: string
          type: array
        run_on:
          enum:
          - first
          - second
          - all
          type: string
      required:
      - plugin
  version: v1
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: kongconsumers.configuration.konghq.com
spec:
  additionalPrinterColumns:
  - JSONPath: .username
    description: Username of a Kong Consumer
    name: Username
    type: string
  - JSONPath: .metadata.creationTimestamp
    description: Age
    name: Age
    type: date
  group: configuration.konghq.com
  names:
    kind: KongConsumer
    plural: kongconsumers
    shortNames:
    - kc
  scope: Namespaced
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      properties:
        credentials:
          items:
            type: string
          type: array
        custom_id:
          type: string
        username:
          type: string
  version: v1
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: kongingresses.configuration.konghq.com
spec:
  group: configuration.konghq.com
  names:
    kind: KongIngress
    plural: kongingresses
    shortNames:
    - ki
  scope: Namespaced
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      properties:
        proxy:
          properties:
            connect_timeout:
              minimum: 0
              type: integer
            path:
              pattern: ^/.*$
              type: string
            protocol:
              enum:
              - http
              - https
              - grpc
              - grpcs
              - tcp
              - tls
              type: string
            read_timeout:
              minimum: 0
              type: integer
            retries:
              minimum: 0
              type: integer
            write_timeout:
              minimum: 0
              type: integer
          type: object
        route:
          properties:
            headers:
              additionalProperties:
                items:
                  type: string
                type: array
              type: object
            https_redirect_status_code:
              type: integer
            methods:
              items:
                type: string
              type: array
            path_handling:
              enum:
              - v0
              - v1
              type: string
            preserve_host:
              type: boolean
            protocols:
              items:
                enum:
                - http
                - https
                - grpc
                - grpcs
                - tcp
                - tls
                type: string
              type: array
            regex_priority:
              type: integer
            request_buffering:
              type: boolean
            response_buffering:
              type: boolean
            snis:
              items:
                type: string
              type: array
            strip_path:
              type: boolean
        upstream:
          properties:
            algorithm:
              enum:
              - round-robin
              - consistent-hashing
              - least-connections
              type: string
            hash_fallback:
              type: string
            hash_fallback_header:
              type: string
            hash_on:
              type: string
            hash_on_cookie:
              type: string
            hash_on_cookie_path:
              type: string
            hash_on_header:
              type: string
            healthchecks:
              properties:
                active:
                  properties:
                    concurrency:
                      minimum: 1
                      type: integer
                    healthy:
                      properties:
                        http_statuses:
                          items:
                            type: integer
                          type: array
                        interval:
                          minimum: 0
                          type: integer
                        successes:
                          minimum: 0
                          type: integer
                      type: object
                    http_path:
                      pattern: ^/.*$
                      type: string
                    timeout:
                      minimum: 0
                      type: integer
                    unhealthy:
                      properties:
                        http_failures:
                          minimum: 0
                          type: integer
                        http_statuses:
                          items:
                            type: integer
                          type: array
                        interval:
                          minimum: 0
                          type: integer
                        tcp_failures:
                          minimum: 0
                          type: integer
                        timeout:
                          minimum: 0
                          type: integer
                      type: object
                  type: object
                passive:
                  properties:
                    healthy:
                      properties:
                        http_statuses:
                          items:
                            type: integer
                          type: array
                        interval:
                          minimum: 0
                          type: integer
                        successes:
                          minimum: 0
                          type: integer
                      type: object
                    unhealthy:
                      properties:
                        http_failures:
                          minimum: 0
                          type: integer
                        http_statuses:
                          items:
                            type: integer
                          type: array
                        interval:
                          minimum: 0
                          type: integer
                        tcp_failures:
                          minimum: 0
                          type: integer
                        timeout:
                          minimum: 0
                          type: integer
                      type: object
                  type: object
                threshold:
                  type: integer
              type: object
            host_header:
              type: string
            slots:
              minimum: 10
              type: integer
          type: object
  version: v1
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: kongplugins.configuration.konghq.com
spec:
  additionalPrinterColumns:
  - JSONPath: .plugin
    description: Name of the plugin
    name: Plugin-Type
    type: string
  - JSONPath: .metadata.creationTimestamp
    description: Age
    name: Age
    type: date
  - JSONPath: .disabled
    description: Indicates if the plugin is disabled
    name: Disabled
    priority: 1
    type: boolean
  - JSONPath: .config
    description: Configuration of the plugin
    name: Config
    priority: 1
    type: string
  group: configuration.konghq.com
  names:
    kind: KongPlugin
    plural: kongplugins
    shortNames:
    - kp
  scope: Namespaced
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      properties:
        config:
          type: object
        configFrom:
          properties:
            secretKeyRef:
              properties:
                key:
                  type: string
                name:
                  type: string
              required:
              - name
              - key
              type: object
          type: object
        disabled:
          type: boolean
        plugin:
          type: string
        protocols:
          items:
            enum:
            - http
            - https
            - grpc
            - grpcs
            - tcp
            - tls
            type: string
          type: array
        run_on:
          enum:
          - first
          - second
          - all
          type: string
      required:
      - plugin
  version: v1
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tcpingresses.configuration.konghq.com
spec:
  additionalPrinterColumns:
  - JSONPath: .status.loadBalancer.ingress[*].ip
    description: Address of the load balancer
    name: Address
    type: string
  - JSONPath: .metadata.creationTimestamp
    description: Age
    name: Age
    type: date
  group: configuration.konghq.com
  names:
    kind: TCPIngress
    plural: tcpingresses
  scope: Namespaced
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      properties:
        apiVersion:
          type: string
        kind:
          type: string
        metadata:
          type: object
        spec:
          properties:
            rules:
              items:
                properties:
                  backend:
                    properties:
                      serviceName:
                        type: string
                      servicePort:
                        format: int32
                        type: integer
                    type: object
                  host:
                    type: string
                  port:
                    format: int32
                    type: integer
                type: object
              type: array
            tls:
              items:
                properties:
                  hosts:
                    items:
                      type: string
                    type: array
                  secretName:
                    type: string
                type: object
              type: array
          type: object
        status:
          type: object
  version: v1beta1
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: kong-serviceaccount
  namespace: kong
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: kong-ingress-clusterrole
rules:
- apiGroups:
  - ""
  resources:
  - endpoints
  - nodes
  - pods
  - secrets
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  - extensions
  - networking.internal.knative.dev
  resources:
  - ingresses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - networking.k8s.io
  - extensions
  - networking.internal.knative.dev
  resources:
  - ingresses/status
  verbs:
  - update
- apiGroups:
  - configuration.konghq.com
  resources:
  - tcpingresses/status
  verbs:
  - update
- apiGroups:
  - configuration.konghq.com
  resources:
  - kongplugins
  - kongclusterplugins
  - kongcredentials
  - kongconsumers
  - kongingresses
  - tcpingresses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - create
  - get
  - update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: kong-ingress-clusterrole-nisa-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kong-ingress-clusterrole
subjects:
- kind: ServiceAccount
  name: kong-serviceaccount
  namespace: kong
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
  name: kong-proxy
  namespace: kong
spec:
  ports:
  - name: proxy
    port: 80
    protocol: TCP
    targetPort: 8000
    nodePort: 31180
  - name: proxy-ssl
    port: 443
    protocol: TCP
    targetPort: 8443
    nodePort: 31144
  selector:
    app: ingress-kong
  type: NodePort
---
apiVersion: v1
kind: Service
metadata:
  name: kong-validation-webhook
  namespace: kong
spec:
  ports:
  - name: webhook
    port: 443
    protocol: TCP
    targetPort: 8080
  selector:
    app: ingress-kong
---
apiVersion: v1
kind: Service
metadata:
  name: kong-proxy-admin
  namespace: kong
spec:
  ports:
  - name: kong-proxy-admin
    port: 8001
    targetPort: 8001
    protocol: TCP
  - name: kong-proxy-admin-ssl
    port: 8444
    targetPort: 8444
    protocol: TCP
  selector:
    app: ingress-kong
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: ingress-kong
  name: ingress-kong
  namespace: kong
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ingress-kong
  template:
    metadata:
      annotations:
        kuma.io/gateway: enabled
        prometheus.io/port: "8100"
        prometheus.io/scrape: "true"
        traffic.sidecar.istio.io/includeInboundPorts: ""
      labels:
        app: ingress-kong
    spec:
      containers:
      - env:
        - name: KONG_PROXY_LISTEN
          value: 0.0.0.0:8000, 0.0.0.0:8443 ssl http2
        - name: KONG_PORT_MAPS
          value: 80:8000, 443:8443
        - name: KONG_ADMIN_LISTEN
          value: 0.0.0.0:8001, 127.0.0.1:8444 ssl
        - name: KONG_STATUS_LISTEN
          value: 0.0.0.0:8100
        - name: KONG_DATABASE
          value: postgres
        - name: KONG_PG_HOST
          value: postgres-service.cgf-ml.svc
        - name: KONG_PG_USER
          value: kong
        - name: KONG_PG_PASSWORD
          value: "123456"
        - name: KONG_PG_DATABASE
          value: kong
        - name: KONG_PG_PORT
          value: "5432"
        - name: KONG_NGINX_WORKER_PROCESSES
          value: "2"
        - name: KONG_ADMIN_ACCESS_LOG
          value: /dev/stdout
        - name: KONG_ADMIN_ERROR_LOG
          value: /dev/stderr
        - name: KONG_PROXY_ERROR_LOG
          value: /dev/stderr
        image: kong:2.1
        lifecycle:
          preStop:
            exec:
              command:
              - /bin/sh
              - -c
              - kong quit
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /status
            port: 8100
            scheme: HTTP
          initialDelaySeconds: 5
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: proxy
        ports:
        - containerPort: 8000
          name: proxy
          protocol: TCP
        - containerPort: 8443
          name: proxy-ssl
          protocol: TCP
        - containerPort: 8100
          name: metrics
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /status
            port: 8100
            scheme: HTTP
          initialDelaySeconds: 5
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
      - env:
        - name: CONTROLLER_KONG_ADMIN_URL
          value: https://127.0.0.1:8444
        - name: CONTROLLER_KONG_ADMIN_TLS_SKIP_VERIFY
          value: "true"
        - name: CONTROLLER_PUBLISH_SERVICE
          value: kong/kong-proxy
        - name: POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        image: kong/kubernetes-ingress-controller:1.2
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 5
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: ingress-controller
        ports:
        - containerPort: 8080
          name: webhook
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 5
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
      serviceAccountName: kong-serviceaccount

四、部署konga

1.迁移konga数据库

// postgresql://<数据库名>:<密码>@ip:端口/konga
docker run --rm pantsel/konga:0.14.9 -c prepare -a postgres -u postgresql://konga:123456@192.168.11.100:30362/konga

2.部署konga

  注意将NO_AUTH设置为false，启用登录界面。

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: konga
  name: konga
  namespace: kong
spec:
  replicas: 1
  selector:
    matchLabels:
      app: konga
  template:
    metadata:
      labels:
        app: konga
    spec:
      containers:
      - env:
        - name: HOST
          value: 0.0.0.0
        - name: PORT
          value: '80'
        - name: NODE_ENV
          value: development
        - name: DB_ADAPTER
          value: postgres
        - name: DB_HOST
          value: postgres-service.cgf-ml.svc
        - name: DB_PORT
          value: '5432'
        - name: DB_USER
          value: konga
        - name: DB_PASSWORD
          value: '123456'
        - name: DB_DATABASE
          value: konga
        - name: DB_PG_SCHEMA
          value: konga
        - name: NO_AUTH
          value: 'false'
        image: pantsel/konga:0.14.9
        name: konga
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: konga
  namespace: kong
spec:
  externalTrafficPolicy: Cluster
  ports:
  - name: konga
    port: 80
    protocol: TCP
    targetPort: 80
    nodePort: 31801
  selector:
    app: konga
  type: NodePort

3.汉化Konga（可选）

https://github.com/jsonljd/konga-lang-plugin

按照操作之后，将相应的目录用hostPath挂载即可

五、Kong的使用

//验证是否安装好kong-admin
curl -i -X GET http://10.43.27.40:8001

//查看services
curl -i -X GET http://10.43.27.40:8001/services/default.httpbin.80

//发布服务
curl pi -X POST http://10.43.27.40:8001/services/ --data name=nginx_service --data url='http://my-nginx.default.svc'

//添加路由
curl -i -X POST http://10.43.27.40:8001/services/nginx_service/routes --data 'paths[]=/nginx' --data 'name=nginx'

//添加速率限制插件
curl -i -X POST http://10.43.27.40:8001/plugins --data "name=rate-limiting" --data "config.minute=5" --data "config.policy=local"

//测试带身份验证的url
curl -i -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyLCJpc3MiOiJ1c2VyMi1qd3QifQ.cH3ABwULs80lwOjLg6OapMd7M4ElScBbc20XnYFzpIs" 192.168.185.95:31180/get
curl -i -X POST --data "foo=bar" -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJjcmVhdGVkX2F0IjoxNjIxNTAzNjM2LCJjb25zdW1lciI6eyJpZCI6IjViMzVhOGUzLWE4MWItNWJkYS1hODkxLWNkYjIzNDg4YjQ5ZSJ9LCJpc3MiO" 192.168.185.95:31180/post

使用yaml文件暴露服务

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: cgf-dataset-ingress
  namespace: cgf-ml
  annotations:  # 只需要加上声明标签即可
    kubernetes.io/ingress.class: kong
spec:
  rules:
  - http:
      paths:
      - path: /k8s/cgf-dataset
        pathType: Prefix
        backend:
          serviceName: cgf-dataset-service
          servicePort: 8001

添加konga插件

# 在ingress的annotations中添加即可
konghq.com/strip-path: "true" # 是否截取访问的url前缀