Ingress-nginx简介
Pod的IP以及service IP只能在集群内访问,如果想在集群外访问kubernetes提供的服务,可以使用nodeport、proxy、loadbalacer以及ingress等方式,由于service的IP集群外不能访问,可以使用ingress方式再代理一次,即ingress代理service,service代理pod。
Ingress基本原理图如下:
官网:https://kubernetes.github.io/ingress-nginx/
源码:https://github.com/kubernetes/ingress-nginx
下面以在阿里云自建kubernetes集群为例,使用SLB做四层代理,转发到ingress-controller节点,基本原理图如下:
备注:在私有云环境可以使用nginx四层代理配合keepalived替换上图LB。
helm部署ingress-nginx
官方参考:
https://kubernetes.github.io/ingress-nginx/deploy/#using-helm
https://github.com/kubernetes/ingress-nginx/blob/master/charts/ingress-nginx/values.yaml
集群节点:
主机名 | IP地址 | 描述 |
---|---|---|
cn-shenzhen.192.168.0.48 | 192.168.0.48 | master节点 |
cn-shenzhen.192.168.0.49 | 192.168.0.49 | ingress节点 |
cn-shenzhen.192.168.0.50 | 192.168.0.50 | ingress节点 |
cn-shenzhen.192.168.0.51 | 192.168.0.51 | node节点 |
部署方式:
- DaemonSet + nodeSeletor
- deployment设置replicas数量 + nodeSeletor + pod互斥
这里选择第一种方式,选择2个worker节点打标签,以daemonset+hostNetwork
方式部署高可用ingress-nginx-controller。
添加helm chat
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm search repo -l ingress-nginx
选择两个worker节点打标签
kubectl label nodes cn-shenzhen.192.168.0.49 node=ingress
kubectl label nodes cn-shenzhen.192.168.0.50 node=ingress
部署ingress-nginx,默认镜像位于国外,需自行想办法保存到可以访问的镜像仓库,这里使用dockerhub上搜索到的镜像:
helm install nginx-ingress ingress-nginx/ingress-nginx \
--namespace ingress-nginx \
--create-namespace \
--set controller.image.registry=willdockerhub \
--set controller.image.image=ingress-nginx-controller \
--set controller.image.tag=v0.48.1 \
--set controller.image.digest="" \
--set controller.hostNetwork=true \
--set controller.kind=DaemonSet \
--set controller.service.type=ClusterIP \
--set controller.hostPort.enable=true \
--set controller.hostPort.http=80 \
--set controller.hostPort.https=443 \
--set controller.nodeSelector.node=ingress
查看创建的pods,被部署在标签为node=ingress
的两个节点上:
[root@master ~]# kubectl -n ingress-nginx get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-ingress-ingress-nginx-controller-5wbfv 1/1 Running 0 4m11s 192.168.0.50 cn-shenzhen.192.168.0.50 <none> <none>
nginx-ingress-ingress-nginx-controller-q9st2 1/1 Running 0 4m11s 192.168.0.49 cn-shenzhen.192.168.0.49 <none> <none>
查看创建的service
[root@master ~]# kubectl -n ingress-nginx get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-ingress-ingress-nginx-controller ClusterIP 172.16.94.159 <none> 80/TCP,443/TCP 4m14s
nginx-ingress-ingress-nginx-controller-admission ClusterIP 172.16.118.78 <none> 443/TCP 4m14s
配置负载均衡
创建带有公网IP的负载均衡实例,监听四层80和443端口,转发到后端两个ingress节点hostNetwork 80和443端口:
配置域名解析,指向SLB公网IP地址:
*apps.cloudcele.com ---> 120.24.77.158
以阿里云域名为例
创建示例应用
创建两副本nginx应用、ClusterIP类型service及ingress规则:
helm repo add bitnami https://charts.bitnami.com/bitnami
helm install nginx-app bitnami/nginx \
--namespace=apps \
--create-namespace \
--set replicaCount=2 \
--set containerPorts.http=8080 \
--set service.type=ClusterIP \
--set service.port=80 \
--set service.targetPortt=8080 \
--set ingress.enabled=true \
--set ingress.pathType=Prefix \
--set ingress.hostname=demo.apps.cloudcele.com \
--set ingress.path=/
查看创建的pods
[root@master ~]# kubectl -n apps get pods
NAME READY STATUS RESTARTS AGE
nginx-app-69c694dd64-9sxsl 1/1 Running 0 13m
nginx-app-69c694dd64-rvn4d 1/1 Running 0 13m
查看创建的service
[root@master ~]# kubectl -n apps get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-app ClusterIP 172.16.215.161 <none> 80/TCP 13m
查看创建的ingress规则
[root@master ~]# kubectl -n apps get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
nginx-app <none> demo.apps.cloudcele.com 172.16.94.159 80 13m
查看ingress规则yaml配置:
[root@master ~]# kubectl -n apps get ingress nginx-app -o yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
meta.helm.sh/release-name: nginx-app
meta.helm.sh/release-namespace: apps
......
spec:
rules:
- host: demo.apps.cloudcele.com
http:
paths:
- backend:
service:
name: nginx-app
port:
name: http
path: /
pathType: Prefix
status:
loadBalancer:
ingress:
- ip: 172.16.94.159
查看ingress运行配置:
[root@master ~]# kubectl -n apps describe ingress nginx-app
Name: nginx-app
Namespace: apps
Address: 172.16.94.159
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
demo.apps.cloudcele.com
/ nginx-app:http (10.9.0.6:8080,10.9.0.74:8080)
Annotations: meta.helm.sh/release-name: nginx-app
meta.helm.sh/release-namespace: apps
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 14m (x2 over 15m) nginx-ingress-controller Scheduled for sync
Normal Sync 14m (x2 over 15m) nginx-ingress-controller Scheduled for sync
使用域名进行访问:
Ingress controller扩容
选择某些worker节点打标签即可:
kubectl label nodes cn-shenzhen.192.168.0.51 node=ingress
由于是daemonset类型,ingress controller pod会自动扩容到打标签的节点
[root@master ~]# kubectl -n ingress-nginx get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-ingress-ingress-nginx-controller-5wbfv 1/1 Running 0 120m 192.168.0.50 cn-shenzhen.192.168.0.50 <none> <none>
nginx-ingress-ingress-nginx-controller-n2g7n 1/1 Running 0 54s 192.168.0.51 cn-shenzhen.192.168.0.51 <none> <none>
nginx-ingress-ingress-nginx-controller-q9st2 1/1 Running 0 120m 192.168.0.49 cn-shenzhen.192.168.0.49 <none> <none>
由多个独占Ingress实例组成统一接入层承载集群入口流量,同时可依据后端业务流量水平扩缩容Ingress节点。当然如果您前期的集群规模并不大,也可以采用将Ingress服务与业务应用混部的方式,但建议进行资源限制和隔离。