Ingress简介
对于基于HTTP的服务来说,不同的URL地址经常对应到不同的后端服务或者虚拟服务器,这些应用层的转发机制仅通过Kubernetes的Service机制是无法实现的。从Kubernetes v1.1版本开始新增Ingress资源对象,用于将不同URL访问请求转发到后端不同的Service,以实现HTTP层的业务路由机制。Kubernetes使用一个Ingress策略定义和一个具体的Ingress Controller,两者结合并实现一个完整的Ingress负载均衡器。
使用Ingress进行负载分发时,Ingress Controller通过api-server监听ingress和service的变化,将基于Ingress策略将客户端请求直接转发到Service对应的后端Endpoint上,这样会跳过kube-proxy的转发功能。
- 未配置ingress:
集群外部 --> NodePort --> k8s Service
- 配置ingress:
集群外部 --> Ingress --> k8s Service
Ingress Controller 除了traefik,还有Haproxy、Linkerd、nginx等其他类型。
Traefik 简介
Traefik 是一个为了让部署微服务更加便捷而诞生的现代HTTP反向代理、负载均衡工具。它支持多种后台(Docker、Swarm、Kubernetes、Mesos、Consul、Etcd、zookeeper等)来自动化、动态的应用他的配置文件来设置。

特性
- 它非常快
- 它无需安装其他依赖,通过Go语音编写的单一可执行文件
- 支持Rest API
- 支持多种后台
- 后台监控,可以监听后台变化而自动化应用新的配置文件设置
- 配置文件热更新,无需重启进程
- 正常结束http连接
- 后端断路器
- 轮询,rebalancer负载均衡
- Rest Metrics
- 支持最小化官方Docker镜像
- 后台支持SSL
- 前台支持SSL
- 清爽的AngularJS前端页面
- 支持WebSocket
- 支持HTTP/2
- 网络错误重试
- 支持Let’s Encrypt(自动更新https证书)
- 高可用集群模式
部署traefik
Traefik可以监听你的服务发现、管理API,并且每当你的的微服务被添加、移除、杀死或者更新都会被感知,并且可以自动生成它们的配置文件。指向到你服务的路由将会被直接创建出来。
安装traefik ingress-controller
因为Traefik需要监听服务发现和管理API,所以需要对Ingress做RBAC授权。
kubecl apply -f traefik-rbac.yaml
# traefik-rbac.yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: traefik-ingress-controller
rules:
- apiGroups:
- ""
resources:
- pods
- services
- endpoints
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- ingresses
verbs:
- get
- list
- watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: traefik-ingress-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
name: traefik-ingress-controller
namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: traefik-ingress-controller
namespace: kube-system
创建ConfigMap 将域名ssl证书挂载到traefik中,将各个域名的SSL证书放到/data/srv/traefik/ssl/目录下。
# kubectl -n kube-system create configmap traefik-ssl --from-file=/data/srv/traefik/ssl/
通过Daemonset和Service将traefik部署到集群的所有节点。 Traefik Pod中80端口为traefik ingress-controller的服务端口,8080端口为traefik的管理WEB界面;为后续配置方便指定80端口暴露NodePort端口为23456,443端口暴露NodePort端口为23457。
通过ConfigMap的形式定义traefik的配置文件traefik.toml,加载各域名SSL证书。
kubectl apply -f traefik-controller.yaml
# traefik-controller.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: traefik-conf
namespace: kube-system
data:
traefik.toml: |
# 设置insecureSkipVerify = true,可以配置backend为443(比如dashboard)的ingress规则
insecureSkipVerify = true
defaultEntryPoints = ["http", "https"]
[entryPoints]
[entryPoints.http]
address = ":80"
### 配置http 强制跳转 https
[entryPoints.http.redirect]
entryPoint = "https"
### 配置只信任trustedIPs传递过来X-Forwarded-*,默认全部信任;为了防止客户端地址伪造,需开启这个
#[entryPoints.http.forwardedHeaders]
# trustedIPs = ["10.1.0.0/16", "172.20.0.0/16", "192.168.1.3"]
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
[[entryPoints.https.tls.certificates]]
CertFile = "/ssl/baijiahulian.pem"
KeyFile = "/ssl/baijiahulian.key"
[[entryPoints.https.tls.certificates]]
CertFile = "/ssl/weishi100.pem"
KeyFile = "/ssl/weishi100.key"
[[entryPoints.https.tls.certificates]]
CertFile = "/ssl/gaotu100.pem"
KeyFile = "/ssl/gaotu100.key"
[[entryPoints.https.tls.certificates]]
CertFile = "/ssl/genshuixue.crt"
KeyFile = "/ssl/genshuixue.key"
[[entryPoints.https.tls.certificates]]
CertFile = "/ssl/umeng100.pem"
KeyFile = "/ssl/umeng100.key"
[[entryPoints.https.tls.certificates]]
CertFile = "/ssl/wenzaizhibo.crt"
KeyFile = "/ssl/wenzaizhibo.key"
[[entryPoints.https.tls.certificates]]
CertFile = "/ssl/babyabc100.pem"
KeyFile = "/ssl/babyabc100.key"
[[entryPoints.https.tls.certificates]]
CertFile = "/ssl/sodareading.pem"
KeyFile = "/ssl/sodareading.key"
---
kind: DaemonSet
apiVersion: apps/v1
metadata:
name: traefik-ingress-controller
namespace: kube-system
labels:
k8s-app: traefik-ingress-lb
spec:
selector:
matchLabels:
k8s-app: traefik-ingress-lb
template:
metadata:
labels:
k8s-app: traefik-ingress-lb
name: traefik-ingress-lb
spec:
serviceAccountName: traefik-ingress-controller
terminationGracePeriodSeconds: 60
volumes:
- name: ssl
configMap:
name: traefik-ssl
- name: config
configMap:
name: traefik-conf
#nodeSelector:
# node-role.kubernetes.io/traefik: "true"
containers:
- image: traefik:v1.7.4
imagePullPolicy: IfNotPresent
name: traefik-ingress-lb
volumeMounts:
- mountPath: "/ssl"
name: "ssl"
- mountPath: "/config"
name: "config"
resources:
limits:
cpu: 1000m
memory: 800Mi
requests:
cpu: 500m
memory: 600Mi
args:
- --configfile=/config/traefik.toml
- --api
- --kubernetes
- --logLevel=INFO
securityContext:
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
ports:
- name: http
containerPort: 80
hostPort: 80
- name: https
containerPort: 443
hostPort: 443
---
kind: Service
apiVersion: v1
metadata:
name: traefik-ingress-service
namespace: kube-system
spec:
selector:
k8s-app: traefik-ingress-lb
ports:
- protocol: TCP
# 该端口为 traefik ingress-controller的服务端口
port: 80
# 集群hosts文件中设置的 NODE_PORT_RANGE 作为 NodePort的可用范围
# 从默认20000~60000之间选一个可用端口,让ingress-controller暴露给外部的访问
nodePort: 23456
name: http
- protocol: TCP
#
port: 443
nodePort: 23457
name: https
- protocol: TCP
# 该端口为 traefik 的管理WEB界面
port: 8080
name: admin
type: NodePort
验证traefik ingress-controller
[root@of-bj-k8s-deploy01 tls]# kubectl get daemonset -n kube-system traefik-ingress-controller
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
traefik-ingress-controller 5 5 5 5 5 <none> 45h
[root@of-bj-k8s-deploy01 tls]# kubectl get svc -n kube-system traefik-ingress-service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
traefik-ingress-service NodePort 10.0.210.214 <none> 80:23456/TCP,443:23457/TCP,8080:21325/TCP 45h
[root@of-bj-k8s-deploy01 tls]#
创建https ingress 例子
创建测试用例
[root@of-bj-k8s-deploy01 tls]# kubectl run test-hello --image=nginx --port=80 --expose
kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
service/test-hello created
deployment.apps/test-hello created
[root@of-bj-k8s-deploy01 tls]# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/test-hello-86f5bff8b4-w6xqh 1/1 Running 0 36s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/test-hello ClusterIP 10.0.200.209 <none> 80/TCP 36s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/test-hello 1/1 1 1 36s
NAME DESIRED CURRENT READY AGE
replicaset.apps/test-hello-86f5bff8b4 1 1 1 36s
在default命名空间创建对应的secret,并为测试用例配置https路由策略。
[root@of-bj-k8s-deploy01 tls]# kubectl create secret tls baijiahulian-cert --key=/data/srv/traefik/ssl/baijiahulian.key --cert=/data/srv/traefik/ssl/baijiahulian.pem
secret/baijiahulian-cert created
[root@of-bj-k8s-deploy01 tls]# cat hello-tls.ing.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-tls-ingress
annotations:
kubernetes.io/ingress.class: traefik
spec:
rules:
- host: hello.baijiahulian.com
http:
paths:
- backend:
serviceName: test-hello
servicePort: 80
tls:
- secretName: baijiahulian-cert
[root@of-bj-k8s-deploy01 tls]# kubectl apply -f hello-tls.ing.yaml
ingress.extensions/hello-tls-ingress created
[root@of-bj-k8s-deploy01 tls]#
创建负载均衡器
在云平台上配置负载均衡器,监听80端口对应集群node节点上的23456端口,监听443端口对应集群node节点上的23457端口。
在客户端hosts增加记录$SLB_IP hello.baijiahulian.com之后,可以在浏览器验证访问。
$ curl -w %{http_code} https://hello.baijiahulian.com
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
200
为Traefik WEB管理页面配置ingress规则
[root@of-bj-k8s-deploy01 traefik]# cat traefik-ui.ing.yaml
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: traefik-web-ui
namespace: kube-system
spec:
rules:
- host: traefik-ui.baijiahulian.com
http:
paths:
- path: /
backend:
serviceName: traefik-ingress-service
servicePort: 8080
[root@of-bj-k8s-deploy01 traefik]# kubectl apply -f traefik-ui.ing.yaml
ingress.extensions/traefik-web-ui created
在客户端增加记录$SLB_IP traefik-ui.baijiahulian.com,并在浏览器进行验证访问。

Traefik 配置基础及全局配置文件
Traefik 官网上已经对Traefik的概念及用法已经做了全面的介绍,本文不在进行具体的描述和配置验证。