K8s网络ingress nginx整理2022版

  最近在学习k8s，有些视频或者资料虽然才一年多，但是，k8s技术更新太快了，所以，学习和怕坑中，先写一篇k8s中比较重的组件ingress
我们采用常用的ingress nginx这种网络方式

1.地址说明

K8s官网：https://kubernetes.io/
版本映射：https://github.com/kubernetes/ingress-nginx
安装版本下载地址：https://github.com/kubernetes/ingress-nginx/releases/tag/controller-v1.1.2
ingress安装说明：https://kubernetes.github.io/ingress-nginx/deploy/
ingress配置说明：https://kubernetes.io/zh/docs/concepts/services-networking/ingress/

2.环境储备

Centos7,k8sm-主节点，k8sn子节点，机器配置就这样了
K8s–1.23.4版本和最基本的kubectl,kubeadm其他最基本的k8s核心组件
Docker–20.10.12版本

3.部署节点

确保nginx-controller运行到k8sn节点上
kubectl label node k8sn name=ingress

4.安装包准备

https://github.com/kubernetes/ingress-nginx/releases/tag/controller-v1.1.2

deploy.yaml文件路径：/root/ingress-nginx-controller-v1.1.2/deploy/static/provider/cloud
deploy.yaml文件修改
查找Deployment关键字，我们把文件修改部分粘贴出来

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  replicas: 1 #只部署一个节点
  minReadySeconds: 0
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/component: controller
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/name: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/component: controller
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
    spec:
      hostNetwork: true ##增加，当前host网络
      containers:
      - args:
        - /nginx-ingress-controller
        - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
        - --election-id=ingress-controller-leader
        - --controller-class=k8s.io/ingress-nginx
        - --ingress-class=nginx
        - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
        - --validating-webhook=:8443
        - --validating-webhook-certificate=/usr/local/certificates/cert
        - --validating-webhook-key=/usr/local/certificates/key
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: LD_PRELOAD
          value: /usr/local/lib/libmimalloc.so
        image: k8s.gcr.io/ingress-nginx/controller:v1.1.2
        imagePullPolicy: IfNotPresent
        lifecycle:
          preStop:
            exec:
              command:
              - /wait-shutdown
		##中间省略。。。。
	 dnsPolicy: ClusterFirst
	      nodeSelector:
	        name: ingress ##选择哪个服务器安装
	        kubernetes.io/os: linux
	      serviceAccountName: ingress-nginx
	      terminationGracePeriodSeconds: 300
	      volumes:
	      - name: webhook-cert
	        secret:
	          secretName: ingress-nginx-admission

5.安装ingress-nginx和问题排查

kubectl apply -f deploy.yaml

kubectl get all -n ingress-nginx

发现 ErrImagePull或者ContainerCreating，但是一直都是这个状态，或者ErrImagePull这个状态都没有，一直时创建中状态

最初以为时节点的443端口占用，查了半天，以为时calico占用，后来看到ErrImagePull才明白问题出在哪。

然后查看下pod的安装情况:
kubectl describe pod/ingress-nginx-admission-create-gdsms -n ingress-nginx

Events:
  Type     Reason     Age                  From               Message
  ----     ------     ----                 ----               -------
  Normal   Scheduled  2m32s                default-scheduler  Successfully assigned ingress-nginx/ingress-nginx-admission-create-gdsms to k8sn
  Warning  Failed     91s                  kubelet            Failed to pull image "k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660": rpc error: code = Unknown desc = Error response from daemon: Get "https://k8s.gcr.io/v2/": dial tcp 142.251.8.82:443: i/o timeout
  Normal   Pulling    65s (x3 over 2m31s)  kubelet            Pulling image "k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660"

发现一直在拉取镜像，都失败了，所以没有创建成功。

所以去网上查了下，有的说配置raw.githubusercontent.com这个网站被污染了，又各种配置hosts的指定IP还是不好使（这种i情况可能老本好使）

那我们有经验就知道了，拉不下来那就去国内的阿里镜像去拉取.

cat deploy.yaml | grep image -C 10
那接下来，我们查询下deploy.yaml有哪些镜像需要拉取：
进入deploy.yaml目录，

查看需要拉取的镜像，去阿里云镜像中心拉取，要不，你在apply的时候，一直都不会成功，而且还不好找问题
一共需要拉取两个镜像：

docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.1.2
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1

**看deploy.yaml中的镜像名字太长了，而且打tag那种格式有问题，**那不如，这样

docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.1.2 k8s.gcr.io/ingress-nginx/controller:v1.1.2
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1 k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1

然后我们去deploy.yaml把对应的镜像名字改掉

一定要注意，你这个时候在k8sm就是部署的主节点（master节点）拉取是没有用的，为啥，因为你的ingress是安装在k8sn(子节点)的，切记
这中间会有一个小问题，命名空间问题，你可以手动创建，现在创建ingress的时候会帮你创建

[root@k8sm deployment_ingress]# kubectl get ns
NAME              STATUS   AGE
default           Active   13d
ingress-nginx     Active   4h7m ## 需要这个命名空间的
kube-node-lease   Active   13d
kube-public       Active   13d
kube-system       Active   13d

删除ingress-nginx: [root@k8sm cloud]# kubectl delete -f deploy.yaml
重新创建：kubectl apply -f deploy.yaml
再看下：kubectl get all -n ingress-nginx
查看pod运行情况

Kubectl get podname -n ingress-nginx -o wide
Kubectl describe podname  -n ingress-nginx

至此，安装成功了，当然，我时删除了至少3次

7.验证

1.创建tomcat,tomcat.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: tomcat-deployment
  labels:
    app: tomcat
spec:
  replicas: 1
  selector:
    matchLabels:
      app: tomcat
  template:
    metadata:
      labels:
        app: tomcat
    spec:
      containers:
      - name: tomcat
        image: tomcat
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: tomcat-service
spec:
  ports:
  - port: 80   
    protocol: TCP
    targetPort: 8080
  selector:
    app: tomcat

查看运行情况
kubectl apply -f tomcat.yaml

kubectl get svc

kubectl get pods

测试下集群内访问

[root@k8sm deployment_ingress]# kubectl get svc -o wide
NAME             TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE     SELECTOR
kubernetes       ClusterIP   10.96.0.1        <none>        443/TCP   12d     <none>
tomcat-service   ClusterIP   10.107.234.209   <none>        80/TCP    2m49s   app=tomcat
[root@k8sm deployment_ingress]# curl 10.107.234.209:80
<!doctype html><html lang="en"><head><title>HTTP Status 404 – Not Found</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 404 – Not Found</h1><hr class="line" /><p><b>Type</b> Status Report</p><p><b>Description</b> The origin server did not find a current representation for the target resource or is not willing to disclose that one exists.</p><hr class="line" /><h3>Apache Tomcat/10.0.14</h3></body></html>[root@k8sm deployment_ingress]# 

为啥会报404呢，因为新版本的tomcat的webapp下部署任何东西

2.创建nginx-ingress.yaml

一定要注意格式，如果不懂可以往上找，官网有说明

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-ingress
spec:
  rules:
  - host: tomcat.lwd.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: tomcat-service
            port:
              number: 80

kubectl apply -f nginx-ingress.yaml

kubectl get ingress

kubectl describe ingress nginx-ingress

3.修改物理win10的hosts文件，添加dns解析

192.168.124.250 tomcat.lwd.com

这块有一个坑，就是你可能认为k8s中网络都是共享的，所以，在配置hosts时，IP可能配置到主节点，注意，这是有问题的，因为你的ingress时安装在k8sn子节点服务器了，ingress也是做外部访问的，所以，网络不是共享的

4打开浏览器，访问tomcat.lwd.com

总结：如果以后想要使用Ingress网络，其实只要定义ingress，service和pod即可，前提是要保证nginx ingress controller已经配置好了。

欢迎大家相互学习