上一回留下一个问题,就是使用浏览器无法访问到,这一回继续排错。 1. 排错思路整理 1) nginx-ingress-controller pod nginx-ingress-controller-6d96ccd6f4-g2d6b , master与node 都可以访问 kubectl describe pod nginx-ingress-controller-6d96ccd6f4-g2d6b -n ingress-nginx Name: nginx-ingress-controller-6d96ccd6f4-g2d6b Namespace: ingress-nginx Node: 192.168.89.133/192.168.89.133 Start Time: Wed, 08 Jan 2020 09:34:34 +0800 Labels: app.kubernetes.io/name=ingress-nginx app.kubernetes.io/part-of=ingress-nginx pod-template-hash=6d96ccd6f4 Annotations: kubernetes.io/limit-ranger: LimitRanger plugin set: cpu, memory request for container nginx-ingress-controller prometheus.io/port: 10254 prometheus.io/scrape: true Status: Running IP: 10.1.20.3 # 这个为nginx-ingress-controller pod 的ip IPs: IP: 10.1.20.3 Controlled By: ReplicaSet/nginx-ingress-controller-6d96ccd6f4 Containers: nginx-ingress-controller: Container ID: docker://20d0722863e8896d830b43f730c8b233009e0a81c9ab1841f805a761ca5d235e Image: 192.168.89.132:5000/nginx-ingress-controller:0.26.2 Image ID: docker-pullable://192.168.89.132:5000/nginx-ingress-controller@sha256:417cf5edc00c1581a44f67c1b1713b55db3811834498f45fc5de40fbe5f6a8de Ports: 80/TCP, 443/TCP Host Ports: 0/TCP, 0/TCP Args: /nginx-ingress-controller --configmap=$(POD_NAMESPACE)/nginx-configuration --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services --udp-services-configmap=$(POD_NAMESPACE)/udp-services --publish-service=$(POD_NAMESPACE)/ingress-nginx --annotations-prefix=nginx.ingress.kubernetes.io State: Running Started: Wed, 08 Jan 2020 14:27:53 +0800 Last State: Terminated Reason: Error Exit Code: 137 Started: Wed, 08 Jan 2020 09:35:24 +0800 Finished: Wed, 08 Jan 2020 14:27:40 +0800 Ready: True Restart Count: 1 Requests: cpu: 100m memory: 90Mi Liveness: http-get http://:10254/healthz delay=10s timeout=10s period=10s #success=1 #failure=3 , 健康性检测 curl http://10.1.20.3:10254/healthz Readiness: http-get http://:10254/healthz delay=0s timeout=10s period=10s #success=1 #failure=3 Environment: POD_NAME: nginx-ingress-controller-6d96ccd6f4-g2d6b (v1:metadata.name) POD_NAMESPACE: ingress-nginx (v1:metadata.namespace) Mounts: /var/run/secrets/kubernetes.io/serviceaccount from nginx-ingress-serviceaccount-token-mz87t (ro) Conditions: Type Status Initialized True Ready True ContainersReady True PodScheduled True Volumes: nginx-ingress-serviceaccount-token-mz87t: Type: Secret (a volume populated by a Secret) SecretName: nginx-ingress-serviceaccount-token-mz87t Optional: false QoS Class: Burstable Node-Selectors: kubernetes.io/os=linux Tolerations: <none> Events: <none> curl http://10.1.20.3:10254/healthz ok 过了一会儿只有89.133这一台可以访问,奇怪! 说明控制器在正常工作,没有问题,不过只有一台可以正常访问接口,不知道这是不是正常的。 kubectl get svc -n ingress-nginx -o wide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR ingress-nginx NodePort 169.169.178.15 <none> 80:30080/TCP,443:30443/TCP 164m app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx service类型为NodePort , service ip : 169.169.178.15 , 该ip两台node可以ping通,master无法ping通 , 该地址为nginx-ingress-controller pod对应的service kubectl describe svc ingress-nginx -n ingress-nginx Name: ingress-nginx Namespace: ingress-nginx Labels: app.kubernetes.io/name=ingress-nginx app.kubernetes.io/part-of=ingress-nginx Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app.kubernetes.io/name":"ingress-nginx","app.kubernetes.io/par... Selector: app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx Type: NodePort IP: 169.169.178.15 Port: http 80/TCP TargetPort: 80/TCP NodePort: http 30080/TCP Endpoints: 10.1.20.3:80 Port: https 443/TCP TargetPort: 443/TCP NodePort: https 30443/TCP Endpoints: 10.1.20.3:443 Session Affinity: None External Traffic Policy: Cluster Events: <none> curl http://169.169.178.15 <html> <head><title>404 Not Found</title></head> <body> <center><h1>404 Not Found</h1></center> <hr><center>openresty/1.15.8.2</center> </body> </html> curl http://169.169.178.15/healthz 回应为空 , 除了 /healthz , 访问其他都报404 2). ingress.yaml 规则 less ingress.yaml spec: # 路由规则 rules: # 主机名,只能是域名,修改为你自己的 - host: k8s.aaa.com http: paths: - path: backend: # 后台部署的 Service Name,与上面部署的 Tomcat 对应 serviceName: tomcat-http # 后台部署的 Service Port,与上面部署的 Tomcat 对应 servicePort: 8080 配置文件最下面部分可知,访问 k8s.aaa.com ,跳转到service名为tomcat-http,端口为8080的服务,思考:这个k8s.aaa.com映射和谁一起? kubectl get Ingress NAME HOSTS ADDRESS PORTS AGE nginx-web k8s.aaa.com 169.169.178.15 80 21h ADDRESS 169.169.178.15为nginx-ingress的服务地址 , 由此可知,Ingress.yaml没有创建容器,而只是创建了一种路由规则,给这个路由规则起了一个名字: nginx-web k8s.aaa.com -> 169.169.178.15 -> 10.1.20.3 -> 起了一个nginx容器,upstream定义了调度规则 -> tomcat-http -> 169.169.70.125(tomcat-http服务ip) -> tomcat-app[ 10.1.27.4(192.168.89.134)/10.1.20.5(192.168.89.133) ] -> 访问到tomcat服务 为了不引起干扰,把master和两台node /etc/hosts里有关k8s.aaa.com的映射都删了。 k8snode01: ip a 6: kube-ipvs0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default link/ether fe:fb:9b:d5:31:d0 brd ff:ff:ff:ff:ff:ff inet 169.169.0.1/32 brd 169.169.0.1 scope global kube-ipvs0 valid_lft forever preferred_lft forever inet 169.169.0.10/32 brd 169.169.0.10 scope global kube-ipvs0 valid_lft forever preferred_lft forever inet 169.169.70.125/32 brd 169.169.70.125 scope global kube-ipvs0 valid_lft forever preferred_lft forever inet 169.169.178.15/32 brd 169.169.178.15 scope global kube-ipvs0 valid_lft forever preferred_lft forever 发现这个ip 169.169.178.15 在192.168.89.133这台机器上被解析在kube-ipvs0上了。 思考: 访问k8s.aaa.com , 就是访问169.169.178.15 , 就是访问192.168.89.133 ? 找一台机器:dnsserver(192.168.89.128): /etc/hosts: 192.168.89.133 k8s.aaa.com curl k8s.aaa.com curl: (7) Failed connect to k8s.aaa.com:80; Connection refused 不行,这样解析不行,咋弄? k8snode01: curl -D- http://169.169.178.15 -H 'Host: k8s.aaa.com' 可以出来 将192.168.89.133 k8s.aaa.com 写到windows hosts 里; 访问加上30080端口, http://k8s.aaa.com:30080/ 终于出来了,访问的时候是要加端口的 3). 为了验证一下是不是这个问题,在测试一个服务 vim test-ingress-pod.yaml apiVersion: v1 kind: Service metadata: name: myapp-svc namespace: default spec: selector: app: myapp env: test ports: - name: http port: 80 targetPort: 80 --- apiVersion: apps/v1 kind: Deployment metadata: name: myapp-test spec: replicas: 2 selector: matchLabels: app: myapp env: test template: metadata: labels: app: myapp env: test spec: containers: - name: myapp image: nginx:1.15-alpine ports: - name: httpd containerPort: 80 vim test-ingress-myapp.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: ingress-myapp namespace: default annotations: kubernetes.io/ingress.class: "nginx" spec: rules: - host: www.tchua.top http: paths: - path: / backend: serviceName: myapp-svc servicePort: 80 kubectl create -f test-ingress-pods.yaml kubectl create -f test-ingress-myapp.yaml windows 里配置 192.168.89.133 www.tchua.top 浏览器访问: http://www.tchua.top:30080 , nginx页面出来了,问题解决 2. 总结 有些问题出现了,一路排错排不出来,可能参考一下其他人的文档,灵感思路就来了。此处要点:这里访问时需要加上svc映射到主机时产生的nodePort端口号,service-nodeport.yaml中定义 - name: http port: 80 targetPort: 80 protocol: TCP nodePort: 30080 - name: https port: 443 targetPort: 443 protocol: TCP nodePort: 30443 有时候就是一个小小的问题,就能卡好几天,一定不要急躁,休息一下,忙点其他东西,再一看,就出来了,另外,对应用的原理还是要特别熟悉,再整理下这个ingress流程: ingress是为了给集群外部暴露服务使用的,首先要部署好ingress controller 控制器,我们选择的是nginx ingress controller,官网上有对应的搭建流程,nginx-ingress-controller镜像比较大,直接构建比较耗时,可以提前下载好导入本地仓库。控制器弄好后,创建测试应用,pod,services等部署好后,再部署ingress.yaml,这个文件编写的是调度规则,给nginx controller控制器使用,要查看ingress controller控制器在哪一台,以及这个服务映射到节点上的端口,进行解析,浏览器访问的时候需要加上端口号。 因为上面为测试用例,如果在真实环境,可以将新的域名解析到公司内部dns服务器上,访问的时候还要使用端口号,这个问题也好解决,在其前部加一个nginx调度,使用标准端口调度到后面的指定的解析端口上即可。 本地应用案例流程: a. 将tchua.top域名解析到公司内部dns服务器上 dnsserver: vim /etc/named.conf ...... zone "tchua.top" IN { type master; file "tchua.top.zone"; }; ...... cp -a /var/named/web.com.zone /var/named/tchua.top.zone vim /var/named/tchua.top.zone $TTL 1D @ IN SOA @ tchua.top. ( 0 ; serial 1D ; refresh 1H ; retry 1W ; expire 3H ) ; minimum NS @ A 192.168.89.133 www A 192.168.89.133 systemctl restart named hosts文件即可去掉,直接浏览器访问 http://www.tchua.top:30080 b. 去掉端口30080访问 修改映射端口http://:30080到常用端口80 vim service-nodeport.yaml ... - name: http port: 80 targetPort: 80 protocol: TCP nodePort: 80 ... kubectl apply -f service-nodeport.yaml kubectl get svc -n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ingress-nginx NodePort 169.169.178.15 <none> 80:80/TCP,443:30443/TCP 5h7m 发现端口已修改 浏览器访问: http://k8s.aaa.com 和 http://www.tchua.top 都可以访问到,验证通过,可以正常使用
云计算之kubernetes服务暴露——ingress 排错
最新推荐文章于 2023-02-03 16:32:20 发布