k8s: ingress基于域名的负载均衡

概念

ingress: 是k8s内部的一个资源对象
ingress controller ingress控制器: 是k8s里启动的一个pod,运行的是nginx的镜像,实现k8s内部的service(ClusterIP类型)的负载均衡
ingress 和ingress controller 的关系:
ingress controller 本质上是一个nginx软件,用来做负载均衡ingress 是k8s内部管理nginx配置(nginx.conf)的组件,用来给ingress controller传参
参考安装部署文档
https://kubernetes.github.io/ingress-nginx/deploy/
使用旧版本ingress controller v1.1完成安装

准备工作

需要提前上传下面的这些镜像和yaml文件到k8s集群里的linux系统里,建议存放到master节点上,然后再scp到node节点上

[root@master .kube]# mkdir /ingress
[root@master .kube]# cd /ingress/
[root@master ingress]# ls
ingress-controller-deploy.yaml         nfs-pvc.yaml                       sc-ingress.yaml
ingress-nginx-controllerv1.1.0.tar.gz  nfs-pv.yaml                        sc-nginx-svc-1.yaml
kube-webhook-certgen-v1.1.0.tar.gz     nginx-deployment-nginx-svc-2.yaml

ingress-controller-deploy.yaml :是部署ingress controller使用的yaml文件
ingress-nginx-controllerv1.1.0.tar.gz : ingress-nginx-controller镜像
kube-webhook-certgen-v1.1.0.tar.gz :kube-webhook-certgen镜像
sc-ingress.yaml 创建ingress的配置文件
sc-nginx-svc-1.yaml : 启动sc-nginx-svc服务和相关pod的yaml
nginx-deployment-nginx-svc-2.yaml : 启动sc-nginx-svc-2服务和相关pod的yaml

步骤1: 安装ingress controller

1.将镜像scp到所有的node节点服务器上

[root@master ingress]# scp ingress-nginx-controllerv1.1.0.tar.gz node-1:/root
ingress-nginx-controllerv1.1.0.tar.gz                                               100%  276MB  42.7MB/s   00:06    
[root@master ingress]# scp ingress-nginx-controllerv1.1.0.tar.gz node-2:/root
ingress-nginx-controllerv1.1.0.tar.gz                                               100%  276MB  45.7MB/s   00:06    
[root@master ingress]# scp kube-webhook-certgen-v1.1.0.tar.gz node-2:/root
kube-webhook-certgen-v1.1.0.tar.gz                                                  100%   47MB  40.5MB/s   00:01    
[root@master ingress]# scp kube-webhook-certgen-v1.1.0.tar.gz node-1:/root
kube-webhook-certgen-v1.1.0.tar.gz                                                  100%   47MB  47.1MB/s   00:00    
[root@master ingress]# 

2.导入镜像,在所有的节点服务器(node-1和node-2)上进行

[root@k8smaster ingress]# docker load -i ingress-nginx-controllerv1.1.0.tar.gz 
[root@k8smaster ingress]# docker load -i kube-webhook-certgen-v1.1.0.tar.gz
[root@k8snode2 ~]# docker images
REPOSITORY                                                                     TAG        IMAGE ID       CREATED         SIZE
registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller   v1.1.0     ae1a7201ec95   16 months ago   285MB
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen       v1.1.1     c41e9fcadf5a   17 months ago   47.7MB
[root@k8smaster new]# 

3.使用ingress-controller-deploy.yaml 文件去启动ingress controller

[root@k8smaster 4-4]# ls
ingress-controller-deploy.yaml         kube-webhook-certgen-v1.1.0.tar.gz  sc-ingress.yaml
ingress-nginx-controllerv1.1.0.tar.gz  nginx-deployment-nginx-svc-2.yaml   sc-nginx-svc-1.yaml
[root@k8smaster 4-4]#

4.执行ingress-controller-deploy.yaml 文件,去启动ingress controller

[root@k8smaster 4-4]# kubectl apply -f ingress-controller-deploy.yaml 
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
configmap/ingress-nginx-controller created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
service/ingress-nginx-controller-admission created
service/ingress-nginx-controller created
deployment.apps/ingress-nginx-controller created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
serviceaccount/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created

5.查看ingress controller的相关命名空间

[root@k8smaster 4-4]# kubectl get ns
NAME              STATUS   AGE
default           Active   11d
ingress-nginx     Active   52s
kube-node-lease   Active   11d
kube-public       Active   11d
kube-system       Active   11d
sanchuang         Active   10d
[root@k8smaster 4-4]# 

6.查看ingress controller的相关service

[root@k8smaster 4-4]# kubectl get svc -n ingress-nginx
NAME                                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.99.160.10   <none>        80:30092/TCP,443:30263/TCP   91s
ingress-nginx-controller-admission   ClusterIP   10.99.138.23   <none>        443/TCP                      91s
[root@k8smaster 4-4]# 

7.查看ingress controller的相关pod

[root@k8smaster 4-4]# kubectl get pod -n ingress-nginx
NAME                                        READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-k69t2        0/1     Completed   0          119s
ingress-nginx-admission-patch-zsrk8         0/1     Completed   1          119s
ingress-nginx-controller-6c8ffbbfcf-bt94p   1/1     Running     0          119s
ingress-nginx-controller-6c8ffbbfcf-d49kx   1/1     Running     0          119s
[root@k8smaster 4-4]# 

步骤2: 创建pod和暴露pod的服务

1.执行sc-nginx-svc-1.yaml文件

root@k8smaster 4-4]# cat sc-nginx-svc-1.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sc-nginx-deploy
  labels:
    app: sc-nginx-feng
spec:
  replicas: 3
  selector:
    matchLabels:
      app: sc-nginx-feng
  template:
    metadata:
      labels:
        app: sc-nginx-feng
    spec:
      containers:
      - name: sc-nginx-feng
        image: nginx
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name:  sc-nginx-svc
  labels:
    app: sc-nginx-svc
spec:
  selector:
    app: sc-nginx-feng
  ports:
  - name: name-of-service-port
    protocol: TCP
    port: 80
    targetPort: 80
[root@k8smaster 4-4]# 
[root@k8smaster 4-4]# kubectl apply -f sc-nginx-svc-1.yaml 
deployment.apps/sc-nginx-deploy created
service/sc-nginx-svc created
[root@k8smaster 4-4]# kubectl get svc
NAME           TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
kubernetes     ClusterIP   10.96.0.1      <none>        443/TCP   11d
sc-nginx-svc   ClusterIP   10.109.14.56   <none>        80/TCP    4s
[root@k8smaster 4-4]# 

2.查看服务器的详细信息,查看Endpoints对应的pod的ip和端口是否正常

[root@k8smaster 4-4]# kubectl describe svc sc-nginx-svc
Name:              sc-nginx-svc
Namespace:         default
Labels:            app=sc-nginx-svc
Annotations:       <none>
Selector:          app=sc-nginx-feng
Type:              ClusterIP
IP Families:       <none>
IP:                10.109.14.56
IPs:               10.109.14.56
Port:              name-of-service-port  80/TCP
TargetPort:        80/TCP
Endpoints:         10.244.185.253:80,10.244.185.254:80,10.244.249.24:80
Session Affinity:  None
Events:            <none>
[root@k8smaster 4-4]# curl 10.109.14.56  访问服务暴露的ip
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@k8smaster 4-4]# 

步骤3: 启用ingress 关联ingress controller 和service

1.执行sc-ingress.yaml文件

[root@k8smaster 4-4]# cat sc-ingress.yaml 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: sc-ingress
  annotations:
    kubernets.io/ingress.class: nginx #注释 这个ingress 是关联ingress controller的
spec:
  ingressClassName: nginx  #关联ingress controller
  rules:
  - host: www.feng.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: sc-nginx-svc
            port:
              number: 80
  - host: www.zhang.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: sc-nginx-svc-2
            port:
              number: 80
[root@k8smaster 4-4]# 
[root@k8smaster 4-4]# kubectl apply -f sc-ingress.yaml   执行文件
ingress.networking.k8s.io/sc-ingress created
[root@k8smaster 4-4]# kubectl get ingress  查看效果
NAME         CLASS   HOSTS                        ADDRESS   PORTS   AGE
sc-ingress   nginx   www.feng.com,www.zhang.com             80      8s
[root@k8smaster 4-4]# 
过几分钟可以看到 有宿主机的ip地址
[root@k8smaster 4-4]# kubectl get ingress
NAME         CLASS   HOSTS                        ADDRESS                       PORTS   AGE
sc-ingress   nginx   www.feng.com,www.zhang.com   192.168.2.222,192.168.2.223   80      52s
[root@k8smaster 4-4]#

步骤4: 查看ingress controller 里的nginx.conf 文件里是否有ingress对应的规则

1.查看业务Pod的名字

[root@k8smaster 4-4]# kubectl get pod -n ingress-nginx
NAME                                        READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-k69t2        0/1     Completed   0          25m
ingress-nginx-admission-patch-zsrk8         0/1     Completed   1          25m
ingress-nginx-controller-6c8ffbbfcf-bt94p   1/1     Running     0          25m
ingress-nginx-controller-6c8ffbbfcf-d49kx   1/1     Running     0          25m
[root@k8smaster 4-4]#

2.进入ingress controller对应的pod里查看nginx.conf的配置

[root@k8smaster 4-4]# kubectl exec -n ingress-nginx -it ingress-nginx-controller-6c8ffbbfcf-bt94p -- bash
bash-5.1$ 
bash-5.1$ cat nginx.conf|grep zhang.com
	## start server www.zhang.com
		server_name www.zhang.com ;
	## end server www.zhang.com
bash-5.1$ cat nginx.conf|grep feng.com
	## start server www.feng.com
		server_name www.feng.com ;
	## end server www.feng.com

bash-5.1$ cat nginx.conf|grep -C3 upstream_balancer
	error_log  /var/log/nginx/error.log notice;
	
	upstream upstream_balancer {
		server 0.0.0.1:1234; # placeholder
		
		balancer_by_lua_block {

3.获取ingress controller对应的service暴露宿主机的端口

访问宿主机和相关端口,就可以验证ingress controller是否能进行负载均衡

[root@k8smaster 4-4]# kubectl get svc -n ingress-nginx
NAME                                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.99.160.10   <none>        80:30092/TCP,443:30263/TCP   37m
ingress-nginx-controller-admission   ClusterIP   10.99.138.23   <none>        443/TCP                      37m
[root@k8smaster 4-4]# 

在其他的宿主机(nfs服务器上)或者windows机器上使用域名进行访问

[root@nfs-server ~]# vim /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.2.222 www.feng.com
192.168.2.223 www.zhang.com

4.访问域名

因为我们是基于域名做的负载均衡的配置,所有必须要在浏览器里使用域名去访问,不能使用ip地址
同时ingress controller做负载均衡的时候是基于http协议的,7层负载均衡

[root@nfs-server ~]# curl  www.feng.com
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@nfs-server ~]# 

访问www.zhang.com出现异常,503错误,是nginx内部错误

[root@nfs-server ~]# curl  www.zhang.com   
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx</center>
</body>
</html>
[root@nfs-server ~]# 

步骤5:启动第2个服务和pod,使用了pv+pvc+nfs

需要提前准备好nfs服务器

1.创建pv和pvc

[root@k8smaster 4-4]# ls
ingress-controller-deploy.yaml         nfs-pvc.yaml                       sc-ingress.yaml
ingress-nginx-controllerv1.1.0.tar.gz  nfs-pv.yaml                        sc-nginx-svc-1.yaml
kube-webhook-certgen-v1.1.0.tar.gz     nginx-deployment-nginx-svc-2.yaml
[root@k8smaster 4-4]# 

[root@k8smaster 4-4]# cat nfs-pv.yaml 
apiVersion: v1
kind: PersistentVolume
metadata:
  name: sc-nginx-pv
  labels:
    type: sc-nginx-pv
spec:
  capacity:
    storage: 10Gi 
  accessModes:
    - ReadWriteMany
  storageClassName: nfs
  nfs:
    path: "/web"       #nfs共享的目录
    server: 192.168.203.131   #nfs服务器的ip地址
    readOnly: false
[root@k8smaster 4-4]# 

提醒:需要检查下nfs服务器是否启动nfs服务,并且设置nfs服务开机启动

[root@nfs-server ~]# service nfs restart
Redirecting to /bin/systemctl restart nfs.service
[root@nfs-server ~]# systemctl enable nfs
[root@nfs-server ~]# 
[root@k8smaster 4-4]# kubectl apply -f nfs-pv.yaml 
persistentvolume/sc-nginx-pv configured
[root@k8smaster 4-4]# 
创建pvc
[root@k8smaster 4-4]# cat nfs-pvc.yaml 
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: sc-nginx-pvc
spec:
  accessModes:
  - ReadWriteMany      
  resources:
     requests:
       storage: 1Gi
  storageClassName: nfs #使用nfs类型的pv
[root@k8smaster 4-4]# 

[root@k8smaster 4-4]# kubectl apply -f nfs-pvc.yaml 
persistentvolumeclaim/sc-nginx-pvc unchanged
[root@k8smaster 4-4]# 

[root@k8smaster 4-4]# kubectl get pv
NAME             CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                   STORAGECLASS   REASON   AGE
sc-nginx-pv      10Gi       RWX            Retain           Bound    default/sc-nginx-pvc    nfs                     20h
task-pv-volume   10Gi       RWO,RWX        Retain           Bound    default/task-pv-claim   manual                  24h
[root@k8smaster 4-4]# kubectl get pvc
NAME            STATUS   VOLUME           CAPACITY   ACCESS MODES   STORAGECLASS   AGE
sc-nginx-pvc    Bound    sc-nginx-pv      10Gi       RWX            nfs            20h
task-pv-claim   Bound    task-pv-volume   10Gi       RWO,RWX        manual         24h
[root@k8smaster 4-4]# 

2.准备启动第2个service和pod

[root@k8smaster 4-4]# cat nginx-deployment-nginx-svc-2.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: sc-nginx-feng-2
  template:
    metadata:
      labels:
        app: sc-nginx-feng-2
    spec:
      volumes:
        - name: sc-pv-storage-nfs
          persistentVolumeClaim:
            claimName: sc-nginx-pvc
      containers:
        - name: sc-pv-container-nfs
          image: nginx
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 80
              name: "http-server"
          volumeMounts:
            - mountPath: "/usr/share/nginx/html"
              name: sc-pv-storage-nfs
---
apiVersion: v1
kind: Service
metadata:
  name:  sc-nginx-svc-2
  labels:
    app: sc-nginx-svc-2
spec:
  selector:
    app: sc-nginx-feng-2
  ports:
  - name: name-of-service-port
    protocol: TCP
    port: 80
    targetPort: 80
[root@k8smaster 4-4]# 

3.应用

[root@k8smaster 4-4]# kubectl apply -f nginx-deployment-nginx-svc-2.yaml 
deployment.apps/nginx-deployment created
service/sc-nginx-svc-2 created
[root@k8smaster 4-4]#

[root@k8smaster 4-4]# kubectl get svc
NAME             TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
kubernetes       ClusterIP   10.96.0.1        <none>        443/TCP   11d
sc-nginx-svc     ClusterIP   10.109.14.56     <none>        80/TCP    48m
sc-nginx-svc-2   ClusterIP   10.104.244.189   <none>        80/TCP    10s
[root@k8smaster 4-4]# 

[root@k8smaster 4-4]# kubectl get svc -n ingress-nginx
NAME                                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.99.160.10   <none>        80:30092/TCP,443:30263/TCP   58m
ingress-nginx-controller-admission   ClusterIP   10.99.138.23   <none>        443/TCP                      58m
[root@k8smaster 4-4]# kubectl get ingress
NAME         CLASS   HOSTS                        ADDRESS                       PORTS   AGE
sc-ingress   nginx   www.feng.com,www.zhang.com   192.168.2.222,192.168.2.223   80      36m
[root@k8smaster 4-4]# 

访问宿主机暴露的端口号30092或者80都可以
使用ingress controller暴露服务,感觉不需要使用30000以上的端口访问,可以直接访问80或者443,比使用service 暴露服务还是有点优势的

[root@nfs-server ~]# curl  www.zhang.com
welcome to sanchuang
welcome to sanchuang  teacherfeng nfs  pv pvc
[root@nfs-server ~]# 

[root@nfs-server ~]# curl  www.feng.com
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@nfs-server ~]# 


搞定

  • 0
    点赞
  • 2
    收藏
    觉得还不错? 一键收藏
  • 1
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值