运维开发实践 - Kubernetes - 从外部访问k8s服务(Ingress Controller部署)

当我们使用Kubernetes的Service时，只有当前集群内的服务可以访问Service信息，那该如何让集群外的用户访问到集群内的应用呢?

1. Service NodePort

通过Service NodePort的形式，我们可以直接将应用通过宿主机的某个特定的端口暴露出去给用户访问；

apiVersion: v1
kind: Pod
metadata:
  name: nginx-nodeport
  labels:
    app: nginx-pod
spec:
  containers:
  - image: nginx
    imagePullPolicy: Always
    name: nginx-pod
    ports:
    - containerPort: 80
      protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-pod
  labels:
    app: nginx-pod
spec:
  type: NodePort
  ports:
  - nodePort: 30010
    targetPort: 80
    protocol: TCP
    port: 80
  selector:
    app: nginx-pod

2. Service LoadBalancer(Ingress)

Ingress 用于公开从集群外部到集群内服务的的HTPP和HTTPS 路由。流量路由由Ingress资源上定义的规则控制；

Ingress 可为Service提供外部可访问的URL，负责均衡流量，终止SSL/TLS，以及基于名称的虚拟托管。Ingress控制器通过负载均衡来实现Ingress，尽管它也可以配置边缘路由器或其他前端来处理流量。

Ingress不会公开任意端口或协议，将HTTP和HTTPS以外的服务公开到Internet时，通常使用Service.Type=NodePort或Service.Type=LoadBalancer类型的Service

2.1. 部署Ingress Controller

ingress controller quickstart

# ingress-nginx controller 1.1.3
 wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.3/deploy/static/provider/cloud/deploy.yaml -O ingress-controller.yaml

sed -i 's/k8s.gcr.io\/ingress-nginx\/controller:v1.1.3@sha256:31f47c1e202b39fadecf822a9b76370bd4baed199a005b3e7d4d1455f4fd3fe2/willdockerhub\/ingress-nginx-controller:v1.1.3@sha256:31f47c1e202b39fadecf822a9b76370bd4baed199a005b3e7d4d1455f4fd3fe2/g' ingress-controller.yaml
 
sed -i 's/k8s.gcr.io\/ingress-nginx\/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660/jettech\/kube-webhook-certgen:v1.1.0/g' ingress-controller.yaml


k delete -n ingress-nginx ValidatingWebhookConfiguration/ingress-nginx-admission

kubectl apply -f ingress-controller.yaml

2.2. 通过Ingress暴露应用

将ingress-controller通过NodePort方式暴露出来

apiVersion: v1
kind: Service
metadata:
  name: ingress-svc-nodeport
  namespace: ingress-nginx
spec:
  type: NodePort
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  ports:
  - nodePort: 30001
    port: 80
    targetPort: 80

这样我们可以访问该nginx服务；

将该ingressclass指定为默认的ingressclass

k edit ingressclass nginx

2.2.1.nginx-deployment

创建ingress时发现抛出校验失败，因此删除该ingress校验钩子；

k delete validatingwebhookconfigurations ingress-nginx-admission

创建nginx deployment并绑定Service

# nginx-deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 8
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
        volumeMounts:
        - name: nginx-volume
          mountPath: /usr/share/nginx/html
      volumes:
      - name: nginx-volume
        hostPath:
          path: "/var/data"
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-deployment-svc
spec:
  selector:
    app: nginx
  ports:
  - name: nginx-deployment-svc-http
    protocol: TCP
    port: 80
    targetPort: 80

2.2.2.为nginx-deployment-svc绑定ingress

# nginx-deployment-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-deployment-svc-ing
  namespace: default
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
  - host: yuanxi.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-deployment-svc
            port:
              number: 80

ingress-nginx 会监听到该ingress的创建，并通过该ingress定义的内容对/etc/nginx/nginx.conf进行配置;(如下图我们可以看到 yuanxi.com 已经被写入了ingress-nginx下的nginx.conf)

2.2.3. 创建tomcat statefulset服务

# tomcat-statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: tomcat-statefulset
spec:
  serviceName: tomcat-svc
  replicas: 5
  selector:
    matchLabels:
      app: tomcat
  template:
    metadata:
      labels:
        app: tomcat
    spec:
      containers:
      - name: tomcat
        image: tomcat
        ports:
        - containerPort: 8080
          name: tomcat-web

2.2.4.为tomcat statefulset绑定tomcat service

创建tomcat statefulset 服务并绑定Service

apiVersion: v1
kind: Service
metadata:
  name: tomcat-headless-svc
  labels:
    app: tomcat
spec:
  ports:
  - port: 8080
    name: tomcat-headless-svc
    targetPort: 8080
  clusterIP: None
  selector:
    app: tomcat
---
# tomcat-statefulset-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: tomcat-svc
  labels:
    app: tomcat
spec:
  selector:
    app: tomcat
  ports:
  - port: 8080
    targetPort: 8080
    protocol: TCP

2.2.5.为tomcat service绑定ingress

# tomcat-statefulset-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tomcat-statefulset-svc-ing
  namespace: default
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  defaultBackend:
    service:
      name: tomcat-svc
      port:
        number: 8080
  rules:
  - host: tomcatliyuan.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: tomcat-svc
            port:
              number: 8080

2.3.部署 LoadBalancer Ingress Controller

# 查看应用ingress 创建情况
k get ingress

ingress address 为空；
ingress-nginx-controller external-ip 状态为pending

因为Kubernetes默认不支持Type为LoadBalancer的服务，因此我们需要手动安装Service LoadBalancer

Kubernetes Ingress Controller
由于Kubernetes本身并不提供LoadBalaner Ingress Controller, 因此通常使用云平台本身的loadBalancer，如果你是自己在裸机上搭建Kubernetes，需要手动搭建LoadBalancer Ingress Controller

MetalLB Installation

wget https://raw.githubusercontent.com/metallb/metallb/v0.13.7/config/manifests/metallb-frr.yaml -O metallb.yaml

kubectl apply -f metallb.yaml

# 用于为Service分配IP
kubectl apply -f - <<EOF
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: first-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.1.240-192.168.1.250
EOF

# L2 network mode 
kubectl apply -f - <<EOF
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  namespace: metallb-system
  name: metallbl2
spec:
  ipAddressPools:
  - first-pool
EOF

# 
kubectl apply -f - <<EOF
apiVersion: metallb.io/v1beta1
kind: BGPAdvertisement
metadata:
  namespace: metallb-system
  name: metallbbgp
EOF

部署LoadBalancer完成后，ingress-nginx-controller就有了external-ip，重新删除部署应用的ingress配置即可生效;


若等待2min该ingress依旧未被分配address地址，请查看ingress-controller Pod日志

2.4. 如何通过自定义的域名访问

因为我们已经通过NodePort Service暴露了ingress-nginx service，因此可直接通过ip:nodeport访问ingress-controller，可以看出，这就是一个nginx应用

# 不难看出，其实nginx只是通过request Header中的Host来转发至不同的Service
curl http://192.168.31.175:30001 -H "Host:yuanxi.com"

配置ip域名映射

由于在tomcat-statefulset-ingress.yaml中配置了defaultBackend，因此找不到host映射的host会默认转发到该service；

Reference

Service Mesh Isito
Kubernetes Ingress
MetalLB LoadBalancer
Ingress controller quickstart

欢迎大家关注公众号[ 从零开始的Go学习 ]，学习运维开发云原生相关技术知识~