Kubernetes Ingress原理和案例（ingress-nginx）

1、ingress-nginx工作原理
ingress contronler通过与k8s的api进行交互，动态的去感知k8s集群中ingress服务规则的变化，然后读取它，并按照定义的ingress规则，转发到k8s集群中对应的service。而这个ingress规则写明了哪个域名对应k8s集群中的哪个service，然后再根据ingress-controller中的nginx配置模板，生成一段对应的nginx配置。然后再把该配置动态的写到ingress-controller的pod里，该ingress-controller的pod里面运行着一个nginx服务，控制器会把生成的nginx配置写入到nginx的配置文件中，然后reload一下，使其配置生效。以此来达到域名分配置及动态更新的效果。

 

2、部署ingress-nginx
通过官网进行安装部署，下载mandatory.yaml和service-nodeport.yaml，将镜像修改为私有镜像。其中mandatory.yaml、service-nodeport.yaml和涉及的镜像文件放在云盘中，地址如下：
链接：https://pan.baidu.com/s/1BhCgq7MH8Lt4CdVq2NXUqw 

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.29.0/deploy/static/mandatory.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.29.0/deploy/static/provider/baremetal/service-nodeport.yaml

[root@k8smaster ingress-nginx]# kubectl create -f mandatory.yaml 
namespace/ingress-nginx created
configmap/nginx-configuration created
configmap/tcp-services created
configmap/udp-services created
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
deployment.apps/nginx-ingress-controller created
limitrange/ingress-nginx created
[root@k8smaster ingress-nginx]#
[root@k8smaster ingress-nginx]# kubectl get pod -n ingress-nginx
NAME                                       READY   STATUS    RESTARTS   AGE
nginx-ingress-controller-86479b5b6-p2bbv   1/1     Running   0          114s
[root@k8smaster ingress-nginx]# kubectl get deployment -n ingress-nginx
NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
nginx-ingress-controller   1/1     1            1           2m3s
[root@k8smaster ingress-nginx]# kubectl get rs -n ingress-nginx
NAME                                 DESIRED   CURRENT   READY   AGE
nginx-ingress-controller-86479b5b6   1         1         1       2m15s
[root@k8smaster ingress-nginx]# kubectl create -f service-nodeport.yaml 
service/ingress-nginx created
[root@k8smaster ingress-nginx]# kubectl get service -n ingress-nginx
NAME            TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx   NodePort   10.99.114.183   <none>        80:32380/TCP,443:31728/TCP   15s
[root@k8smaster ingress-nginx]# 

3、http代理访问
[root@k8smaster ingress1]# more tomcat1-deployment.yaml 
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: tomcat1-deployment
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: tomcat1
    spec:
      containers:
      - name: tomcat1
        image: 192.168.23.100:5000/tomcat:v2
        ports:
        - containerPort: 8080
[root@k8smaster ingress1]# more tomcat1-service.yaml 
apiVersion: v1
kind: Service
metadata:
  name: tomcat1-service
spec:
  selector:
    app: tomcat1
  ports:
  - protocol: TCP
    port: 8080
    targetPort: 8080
[root@k8smaster ingress1]# more ingress1.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-tomcat
spec:
  rules:
  - host: www1.ingress.com
    http:
      paths:
      - path: /
        backend:
          serviceName: tomcat1-service
          servicePort: 8080
[root@k8smaster ingress1]# 
[root@k8smaster ingress-nginx]# kubectl apply -f tomcat1-deployment.yaml 
deployment.extensions/tomcat1-deployment created
[root@k8smaster ingress-nginx]# kubectl get deployment
NAME                 READY   UP-TO-DATE   AVAILABLE   AGE
tomcat1-deployment   1/1     1            1           8s
[root@k8smaster ingress-nginx]# kubectl get pod -o wide
NAME                                  READY   STATUS    RESTARTS   AGE   IP            NODE        NOMINATED NODE   READINESS GATES
tomcat1-deployment-6b7859755c-zssfp   1/1     Running   0          17s   10.244.2.18   k8snode02   <none>           <none>
[root@k8smaster ingress-nginx]# curl "http://10.244.2.18:8080"
tomcat1-deployment-6b7859755c-zssfp
Hello world
[root@k8smaster ingress-nginx]# kubectl create -f tomcat1-service.yaml 
service/tomcat1-service created
[root@k8smaster ingress-nginx]# kubectl get service
NAME              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
kubernetes        ClusterIP   10.96.0.1       <none>        443/TCP    4d10h
tomcat1-service   ClusterIP   10.102.163.44   <none>        8080/TCP   6s
[root@k8smaster ingress-nginx]# 
[root@k8smaster ingress-nginx]# curl "http://10.102.163.44:8080"
tomcat1-deployment-6b7859755c-zssfp
Hello world
[root@k8smaster ingress-nginx]# 
[root@k8smaster ingress-nginx]# kubectl create -f ingress1.yaml 
ingress.extensions/ingress-tomcat created
[root@k8smaster ingress-nginx]# kubectl get ingress
NAME             HOSTS              ADDRESS         PORTS   AGE
ingress-tomcat   www1.ingress.com   10.99.114.183   80      13m
[root@k8smaster ingress-nginx]# kubectl get service -n ingress-nginx
NAME            TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx   NodePort   10.99.114.183   <none>        80:32380/TCP,443:31728/TCP   65m
[root@k8smaster ingress-nginx]# kubectl describe ingress ingress-tomcat
Name:             ingress-tomcat
Namespace:        default
Address:          10.99.114.183
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host              Path  Backends
  ----              ----  --------
  www1.ingress.com  
                    /   tomcat1-service:8080 (10.244.2.18:8080)
Annotations:
Events:
  Type    Reason  Age        From                      Message
  ----    ------  ----       ----                      -------
  Normal  CREATE  <invalid>  nginx-ingress-controller  Ingress default/ingress-tomcat
  Normal  UPDATE  <invalid>  nginx-ingress-controller  Ingress default/ingress-tomcat
[root@k8smaster ingress-nginx]#
[root@k8smaster ingress-nginx]# echo "192.168.23.100 www1.ingress.com" >>/etc/hosts
[root@k8smaster ingress-nginx]# curl "http://www1.ingress.com:32380"
tomcat1-deployment-6b7859755c-zssfp
Hello world

[root@k8smaster ingress-nginx]# kubectl get pod -n ingress-nginx
NAME                                       READY   STATUS    RESTARTS   AGE
nginx-ingress-controller-86479b5b6-p2bbv   1/1     Running   1          63m
[root@k8smaster ingress-nginx]# kubectl exec -it nginx-ingress-controller-86479b5b6-p2bbv -n ingress-nginx -- /bin/sh
/etc/nginx $ more nginx.conf  #nginx.conf会更新后端配置

4、https代理访问
创建证书格式：
kubectl create secret tls ${CERT_NAME} --key ${KEY_FILE} --cert ${CERT_FILE}
官网配置方法：https://kubernetes.github.io/ingress-nginx/user-guide/tls/

 

[root@k8smaster https]# openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=nginxsvc/O=nginxsvc"
Generating a 2048 bit RSA private key
...........+++
....+++
writing new private key to 'tls.key'
-----
[root@k8smaster https]# ls -lrt
total 8
-rw-r--r-- 1 root root 1704 Feb 16 16:27 tls.key
-rw-r--r-- 1 root root 1143 Feb 16 16:27 tls.crt
[root@k8smaster https]# 
[root@k8smaster https]# kubectl create secret tls tls-secret --key tls.key --cert tls.crt
secret/tls-secret created
[root@k8smaster https]# more ingress-https.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-tomcat-https
spec:
  tls:
  - hosts:
    - wwws.ingress.com
    secretName: tls-secret
  rules:
  - host: wwws.ingress.com
    http:
      paths:
      - path: /
        backend:
          serviceName: tomcat1-service
          servicePort: 8080
[root@k8smaster https]# kubectl create -f ingress-https.yaml 
ingress.extensions/ingress-tomcat-https created
[root@k8smaster https]# echo "192.168.23.100 wwws.ingress.com" >>/etc/hosts
[root@k8smaster https]# curl "https://wwws.ingress.com:31728"
curl: (60) Peer's Certificate has expired.
More details here: http://curl.haxx.se/docs/sslcerts.html
 [root@k8smaster https]# curl "https://wwws.ingress.com:31728" -Lk
tomcat1-deployment-6b7859755c-zssfp
Hello world
[root@k8smaster https]# 

5、nginx进行BasicAuth(依赖httpd)
官网配置方法：
https://kubernetes.github.io/ingress-nginx/examples/auth/basic/

[root@k8smaster auth]# htpasswd -c auth foo
New password: 
Re-type new password: 
Adding password for user foo
[root@k8smaster auth]# ls -lrt
total 4
-rw-r--r-- 1 root root 42 Feb 16 16:51 auth
[root@k8smaster auth]# 
[root@k8smaster auth]# kubectl create secret generic basic-auth --from-file=auth
secret/basic-auth created
[root@k8smaster auth]# kubectl get secret basic-auth -o yaml
apiVersion: v1
data:
  auth: Zm9vOiRhcHIxJExyT2lycDFsJE8wVkxHSXJZTXlYZkUwbjI3OVk5Ti4K
kind: Secret
metadata:
  creationTimestamp: "2020-02-16T08:51:48Z"
  name: basic-auth
  namespace: default
  resourceVersion: "557306"
  selfLink: /api/v1/namespaces/default/secrets/basic-auth
  uid: 44518018-2b7c-4995-b006-2580c59ba957
type: Opaque
[root@k8smaster auth]# 
[root@k8smaster auth]# more ingress-auth.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-tomcat-auth
  annotations:
    # type of authentication
    nginx.ingress.kubernetes.io/auth-type: basic
    # name of the secret that contains the user/password definitions
    nginx.ingress.kubernetes.io/auth-secret: basic-auth
    # message to display with an appropriate context why the authentication is required
    nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - foo'
spec:
  rules:
  - host: wwwauth.ingress.com
    http:
      paths:
      - path: /
        backend:
          serviceName: tomcat1-service
          servicePort: 8080
[root@k8smaster auth]# kubectl create -f ingress-auth.yaml 
ingress.extensions/ingress-tomcat-auth created
[root@k8smaster https]# echo "192.168.23.100 wwwauth.ingress.com" >>/etc/hosts
[root@k8smaster auth]# curl "http://wwwauth.ingress.com:32380" #访问拦截
<html>
<head><title>401 Authorization Required</title></head>
<body>
<center><h1>401 Authorization Required</h1></center>
<hr><center>nginx/1.17.8</center>
</body>
</html>

[root@k8smaster auth]# curl -v "http://wwwauth.ingress.com:32380" -u 'foo:123456' #需要密码访问
* About to connect() to wwwauth.ingress.com port 32380 (#0)
*   Trying 192.168.23.100...
* Connected to wwwauth.ingress.com (192.168.23.100) port 32380 (#0)
* Server auth using Basic with user 'foo'
> GET / HTTP/1.1
> Authorization: Basic Zm9vOjEyMzQ1Ng==
> User-Agent: curl/7.29.0
> Host: wwwauth.ingress.com:32380
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx/1.17.8
< Date: Mon, 17 Feb 2020 11:13:17 GMT
< Content-Type: text/html
< Content-Length: 48
< Connection: keep-alive
< Accept-Ranges: bytes
< ETag: W/"48-1581932159000"
< Last-Modified: Mon, 17 Feb 2020 09:35:59 GMT
< 
tomcat1-deployment-6b7859755c-zssfp
Hello world
* Connection #0 to host wwwauth.ingress.com left intact
[root@k8smaster auth]#

6、nginx重新
官网配置方法：https://kubernetes.github.io/ingress-nginx/examples/rewrite/

Name	Description	Values
nginx.ingress.kubernetes.io/rewrite-target	Target URI where the traffic must be redirected(必须重定向流量的目标URI)	string
nginx.ingress.kubernetes.io/ssl-redirect	Indicates if the location section is accessible SSL only (defaults to True when Ingress contains a Certificate)	bool
nginx.ingress.kubernetes.io/force-ssl-redirect	Forces the redirection to HTTPS even if the Ingress is not TLS Enabled	bool
nginx.ingress.kubernetes.io/app-root	Defines the Application Root that the Controller must redirect if it's in '/' context	string
nginx.ingress.kubernetes.io/use-regex	Indicates if the paths defined on an Ingress use regular expressions	bool

 

[root@k8smaster rewrite]# more ingress-rewrite.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-tomcat-rewrite
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: http://www1.ingress.com:32380
spec:
  rules:
  - host: re.ingress.com
    http:
      paths:
      - path: /
        backend:
          serviceName: tomcat1-service
          servicePort: 8080
[root@k8smaster rewrite]#
[root@k8smaster rewrite]# echo "192.168.23.100 re.ingress.com" >>/etc/hosts
[root@k8smaster rewrite]# curl -v "http://re.ingress.com:32380"
* About to connect() to re.ingress.com port 32380 (#0)
*   Trying 192.168.23.100...
* Connected to re.ingress.com (192.168.23.100) port 32380 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: re.ingress.com:32380
> Accept: */*
> 
< HTTP/1.1 302 Moved Temporarily
< Server: nginx/1.17.8
< Date: Mon, 17 Feb 2020 15:23:24 GMT
< Content-Type: text/html
< Content-Length: 145
< Connection: keep-alive
< Location: http://www1.ingress.com:32380
< 
<html>
<head><title>302 Found</title></head>
<body>
<center><h1>302 Found</h1></center>
<hr><center>nginx/1.17.8</center>
</body>
</html>