ingress介绍
ingress controller 本质上是一个nginx软件,用来做负载均衡。
ingress 是k8s内部管理nginx配置(nginx.conf)的组件,用来给ingress controller传参。
使用yaml文件方式去安装部署ingress controller
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.7.0/deploy/static/provider/cloud/deploy.yaml
所有需要用到的文件介绍
[root@k8smaster new]# ls
ingress-controller-deploy.yaml nginx-deployment-nginx-svc-2.yaml
ingress-nginx-controllerv1.1.0.tar.gz sc-ingress.yaml
kube-webhook-certgen-v1.1.0.tar.gz sc-nginx-svc-1.yaml
ingress-controller-deploy.yaml # 是部署ingress controller使用的yaml文件
ingress-nginx-controllerv1.1.0.tar.gz # ingress-nginx-controller镜像
kube-webhook-certgen-v1.1.0.tar.gz # kube-webhook-certgen镜像
sc-ingress.yaml # 创建ingress的配置文件
sc-nginx-svc-1.yaml # 启动sc-nginx-svc服务和相关pod的yaml
nginx-deployment-nginx-svc-2.yaml # 启动sc-nginx-svc-2服务和相关pod的yaml
第1大步骤:安装ingress controller
1.将镜像scp到所有的node节点服务器上
[root@k8smaster ingress]# scp ingress-nginx-controllerv1.1.0.tar.gz k8snode1:/root
ingress-nginx-controllerv1.1.0.tar.gz 100% 276MB 101.1MB/s 00:02
[root@k8smaster ingress]# scp ingress-nginx-controllerv1.1.0.tar.gz k8snode2:/root
ingress-nginx-controllerv1.1.0.tar.gz 100% 276MB 98.1MB/s 00:02
[root@k8smaster ingress]# scp kube-webhook-certgen-v1.1.0.tar.gz k8snode1:/root
kube-webhook-certgen-v1.1.0.tar.gz 100% 47MB 93.3MB/s 00:00
[root@k8smaster ingress]# scp kube-webhook-certgen-v1.1.0.tar.gz k8snode2:/root
kube-webhook-certgen-v1.1.0.tar.gz 100% 47MB 39.3MB/s 00:01
2.导入镜像,在所有的节点服务器上进行
[root@k8snode1 ~]# docker load -i ingress-nginx-controllerv1.1.0.tar.gz
[root@k8snode1 ~]# docker load -i kube-webhook-certgen-v1.1.0.tar.gz
[root@k8snode1 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller v1.1.0 ae1a7201ec95 16 months ago 285MB
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen v1.1.1 c41e9fcadf5a 17 months ago 47.7MB
[root@k8snode2 ~]# docker load -i ingress-nginx-controllerv1.1.0.tar.gz
[root@k8snode2 ~]# docker load -i kube-webhook-certgen-v1.1.0.tar.gz
[root@k8snode2 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller v1.1.0 ae1a7201ec95 16 months ago 285MB
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen v1.1.1 c41e9fcadf5a 17 months ago 47.7MB
3.使用ingress-controller-deploy.yaml 文件去启动ingress controller
[root@k8smaster new]# kubectl apply -f ingress-controller-deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
configmap/ingress-nginx-controller created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
service/ingress-nginx-controller-admission created
service/ingress-nginx-controller created
deployment.apps/ingress-nginx-controller created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
serviceaccount/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
查看ingress controller的相关命名空间
[root@k8smaster new]# kubectl get ns
NAME STATUS AGE
default Active 11d
ingress-nginx Active 54s
kube-node-lease Active 11d
kube-public Active 11d
kube-system Active 11d
查看ingress controller的相关service
[root@k8smaster new]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 10.98.50.192 <none> 80:31240/TCP,443:32563/TCP 92s
ingress-nginx-controller-admission ClusterIP 10.97.247.60 <none> 443/TCP 93s
查看ingress controller的相关pod
[root@k8smaster new]# kubectl get pod -n ingress-nginx
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-mcjp8 0/1 Completed 0 2m12s
ingress-nginx-admission-patch-bh6hw 0/1 Completed 0 2m12s
ingress-nginx-controller-6c8ffbbfcf-zdmq8 1/1 Running 0 2m12s
ingress-nginx-controller-6c8ffbbfcf-zvbp2 1/1 Running 0 2m12s
第2大步骤:创建pod和暴露pod的服务
[root@k8smaster new]# cat sc-nginx-svc-1.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: sc-nginx-deploy
labels:
app: sc-nginx-feng
spec:
replicas: 3
selector:
matchLabels:
app: sc-nginx-feng
template:
metadata:
labels:
app: sc-nginx-feng
spec:
containers:
- name: sc-nginx-feng
image: nginx
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: sc-nginx-svc
labels:
app: sc-nginx-svc
spec:
selector:
app: sc-nginx-feng
ports:
- name: name-of-service-port
protocol: TCP
port: 80
targetPort: 80
[root@k8smaster new]# kubectl apply -f sc-nginx-svc-1.yaml
deployment.apps/sc-nginx-deploy created
service/sc-nginx-svc created
[root@k8smaster new]# kubectl get pod
NAME READY STATUS RESTARTS AGE
sc-nginx-deploy-7bb895f9f5-h957k 1/1 Running 0 5s
sc-nginx-deploy-7bb895f9f5-kcndx 1/1 Running 0 5s
sc-nginx-deploy-7bb895f9f5-w9vbn 1/1 Running 0 5s
[root@k8smaster new]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 11d
sc-nginx-svc ClusterIP 10.107.184.147 <none> 80/TCP 48s
查看服务器的详细信息,查看Endpoints对应的pod的ip和端口是否正常
[root@k8smaster new]# kubectl describe svc sc-nginx-svc
Name: sc-nginx-svc
Namespace: default
Labels: app=sc-nginx-svc
Annotations: <none>
Selector: app=sc-nginx-feng
Type: ClusterIP
IP Families: <none>
IP: 10.107.184.147
IPs: 10.107.184.147
Port: name-of-service-port 80/TCP
TargetPort: 80/TCP
Endpoints: 10.244.185.195:80,10.244.185.197:80,10.244.249.34:80
Session Affinity: None
Events: <none>
访问服务暴露的ip
[root@k8smaster new]# curl 10.107.184.147
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
第3大步骤: 启用ingress关联ingress controller 和service
[root@k8smaster new]# cat sc-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: sc-ingress
annotations:
kubernets.io/ingress.class: nginx #注释 这个ingress 是关联ingress controller的
spec:
ingressClassName: nginx #关联ingress controller
rules:
- host: www.feng.com
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: sc-nginx-svc
port:
number: 80
- host: www.zhang.com
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: sc-nginx-svc-2
port:
number: 80
[root@k8smaster new]# kubectl apply -f sc-ingress.yaml
ingress.networking.k8s.io/sc-ingress created
查看效果
[root@k8smaster new]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
sc-ingress nginx www.feng.com,www.zhang.com 192.168.102.137,192.168.102.138 80 79s
第4步:查看ingress controller 里的nginx.conf 文件里是否有ingress对应的规则
[root@k8smaster new]# kubectl get pod -n ingress-nginx
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-mcjp8 0/1 Completed 0 110m
ingress-nginx-admission-patch-bh6hw 0/1 Completed 0 110m
ingress-nginx-controller-6c8ffbbfcf-zdmq8 1/1 Running 0 110m
ingress-nginx-controller-6c8ffbbfcf-zvbp2 1/1 Running 0 110m
进入ingress controller对应的pod里查看nginx.conf的配置
[root@k8smaster new]# kubectl exec -n ingress-nginx -it ingress-nginx-controller-6c8ffbbfcf-zdmq8 -- bash
bash-5.1$ cat nginx.conf|grep zhang.com
## start server www.zhang.com
server_name www.zhang.com ;
## end server www.zhang.com
bash-5.1$ cat nginx.conf|grep feng.com
## start server www.feng.com
server_name www.feng.com ;
## end server www.feng.com
bash-5.1$ cat nginx.conf|grep -C3 upstream_balancer
error_log /var/log/nginx/error.log notice;
upstream upstream_balancer {
server 0.0.0.1:1234; # placeholder
balancer_by_lua_block {
获取ingress controller对应的service暴露宿主机的端口,访问宿主机和相关端口,就可以验证ingress controller是否能进行负载均衡
[root@k8smaster new]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 10.98.50.192 <none> 80:31240/TCP,443:32563/TCP 112m
ingress-nginx-controller-admission ClusterIP 10.97.247.60 <none> 443/TCP 112m
在其他的宿主机或者windows机器上使用域名进行访问
[root@k8smaster new]# vim /etc/hosts
[root@k8smaster new]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.102.137 www.feng.com
192.168.102.138 www.zhang.com
因为我们是基于域名做的负载均衡的配置,所有必须要在浏览器里使用域名去访问,不能使用ip地址
同时ingress controller做负载均衡的时候是基于http协议的,7层负载均衡
[root@k8smaster new]# curl www.feng.com
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
第5步:启动第2个服务和pod,使用了pv+pvc+nfs
需要提前准备好nfs服务器+创建pv和pvc
[root@k8smaster 4-4]# ls
ingress-controller-deploy.yaml nfs-pvc.yaml sc-ingress.yaml
ingress-nginx-controllerv1.1.0.tar.gz nfs-pv.yaml sc-nginx-svc-1.yaml
kube-webhook-certgen-v1.1.0.tar.gz nginx-deployment-nginx-svc-2.yaml
[root@k8smaster 4-4]# cat nfs-pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: sc-nginx-pv
labels:
type: sc-nginx-pv
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteMany
storageClassName: nfs
nfs:
path: "/sc/web" #nfs共享的目录
server: 192.168.2.136 #nfs服务器的ip地址
readOnly: false
[root@k8smaster 4-4]# kubectl apply -f nfs-pv.yaml
persistentvolume/sc-nginx-pv configured
创建pvc
[root@k8smaster 4-4]# cat nfs-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: sc-nginx-pvc
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
storageClassName: nfs #使用nfs类型的pv
[root@k8smaster 4-4]# kubectl apply -f nfs-pvc.yaml
persistentvolumeclaim/sc-nginx-pvc unchanged
[root@k8smaster new]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
sc-nginx-pv 10Gi RWX Retain Bound default/sc-nginx-pvc nfs 11h
task-pv-volume 10Gi RWO Retain Bound default/task-pv-claim manual 12h
[root@k8smaster new]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
sc-nginx-pvc Bound sc-nginx-pv 10Gi RWX nfs 11h
task-pv-claim Bound task-pv-volume 10Gi RWO manual 11h
www-web-0 Pending my-storage-class 5d
启动pod
[root@k8smaster new]# cat nginx-deployment-nginx-svc-2.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: sc-nginx-feng-2
template:
metadata:
labels:
app: sc-nginx-feng-2
spec:
volumes:
- name: sc-pv-storage-nfs
persistentVolumeClaim:
claimName: sc-nginx-pvc
containers:
- name: sc-pv-container-nfs
image: nginx
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
name: "http-server"
volumeMounts:
- mountPath: "/usr/share/nginx/html"
name: sc-pv-storage-nfs
---
apiVersion: v1
kind: Service
metadata:
name: sc-nginx-svc-2
labels:
app: sc-nginx-svc-2
spec:
selector:
app: sc-nginx-feng-2
ports:
- name: name-of-service-port
protocol: TCP
port: 80
targetPort: 80
[root@k8smaster new]# kubectl apply -f nginx-deployment-nginx-svc-2.yaml
deployment.apps/nginx-deployment created
service/sc-nginx-svc-2 created
[root@k8smaster new]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 10.98.50.192 <none> 80:31240/TCP,443:32563/TCP 112m
ingress-nginx-controller-admission ClusterIP 10.97.247.60 <none> 443/TCP 112m
[root@k8smaster new]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
sc-ingress nginx www.feng.com,www.zhang.com 192.168.102.137,192.168.102.138 80 96m
访问宿主机暴露的端口号30092或者80都可以
使用ingress controller暴露服务,感觉不需要使用30000以上的端口访问,可以直接访问80或者443
比使用service 暴露服务还是有点优势
[root@k8smaster new]# curl www.zhang.com
welcome to sanchuang
teacher feng nfs pv pvc