helm部署ingress-nginx

sj1163739403

于 2024-09-29 16:05:17 发布

阅读量595

点赞数 16

分类专栏： Kubernetes 文章标签： nginx 运维 kubernetes

本文链接：https://blog.csdn.net/sj1163739403/article/details/142637731

版权

Kubernetes 专栏收录该内容

10 篇文章 0 订阅

订阅专栏

helm部署ingress-nginx
19/100
保存草稿
发布文章
sj1163739403
未选择任何文件
ingress-nginx是管理南北向流量很好的一个工具，可以避免在云端频繁对LB进行配置，通过Label也可以实现让ingress-nginx都部署在指定的NodeGroup

一、helm安装ingress-nginx

版本选择，先确认要使用的版本
https://github.com/kubernetes/ingress-nginx

1.1、ingress-nginx vaules文件配置

在这里插入图片描述

kind: DaemonSet
NodeSelector:
  nginx: "true"
  type: NodePort
  nodePorts:
    http: "30080"
    https: "30443"
# 创建了pvc存放日志
  extraVolumeMounts:
    - name: log-volume
      mountPath: /var/log/nginx

  # -- Additional volumes to the controller pod.
  extraVolumes:
    - name: log-volume
      persistentVolumeClaim:
        claimName: ingress-nginx-pvc

1.2、安装

helm install nginx ingress-nginx/ingress-nginx --namespace ingress-nginx

1.3、给ingress node配置label

NodeSelector:
  nginx: "true"

kubectl label node ip-172-28-68-223.ap-southeast-1.compute.internal nginx=true

1.4、获取客户端真实IP

这里配置了forwarded-ip和日志格式

data:
  allow-snippet-annotations: "false"
  compute-full-forwarded-for: "true"
  log-format-upstream: '{"@timestamp":"$time_iso8601","host":"$hostname","server_ip":"$server_addr","client_ip":"$http_x_forwarded_for","xff":"$http_x_forwarded_for","domain":"$host","url":"$uu
ri","referer":"$http_referer","args":"$args","upstreamtime":"$upstream_response_time","responsetime":"$request_time","request_method":"$request_method","status":"$status","size":"$body_bytes_ss
ent","request_length":"$request_length","protocol":"$server_protocol","upstreamhost":"$upstream_addr","file_dir":"$request_filename","http_user_agent":"$http_user_agent"}'
  use-forwarded-headers: "true

进入容器后查看access.log
真实的客户端IP存储在http_x_forwarded_for字段中

{"@timestamp":"2024-09-29T07:25:31+00:00","host":"nginx-ingress-nginx-controller-nz9hq","server_ip":"172.28.2.239","client_ip":"xxxxxxxxx, 172.69.63.211","

配置ingress资源

我从kubernetes1.18直接升级到了1.29 1.30，ingress-nginx的版本也是大变样，语法也有了一定变化
ingressClassName不在是annotions配置，属于spec资源，另外host和path也有变化，有个模板配置以后问题不大

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: vault-ui
  namespace: vault
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
  - host: vault.baga.life
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: vault
            port:
              number: 8200

三、配置grpc ngress资源

参考文档
https://help.aliyun.com/zh/ack/ack-managed-and-ack-dedicated/user-guide/use-an-ingress-controller-to-access-grpc-services

3.1、下载grpcurl工具

wget https://github.com/fullstorydev/grpcurl/releases/download/v1.8.7/grpcurl_1.8.7_linux_x86_64.tar.gz

3.2、dp和service配置

grpc-dp.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: grpc-service
  namespace: baga
spec:
  replicas: 1
  selector:
    matchLabels:
      run: grpc-service
  template:
    metadata:
      labels:
        run: grpc-service
    spec:
      containers:
      - image: registry.cn-hangzhou.aliyuncs.com/acs-sample/grpc-server:latest
        imagePullPolicy: Always
        name: grpc-service
        ports:
        - containerPort: 50051
          protocol: TCP
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: grpc-service
  namespace: baga
spec:
  ports:
  - port: 50051
    protocol: TCP
    targetPort: 50051
  selector:
    run: grpc-service
  sessionAffinity: None
  type: ClusterIP

3.3、grpc ingress配置

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: grpc-ingress
  namespace: baga
  annotations:
    # 必须指明后端服务为gRPC服务,核心配置
    nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
spec:
  ingressClassName: nginx
  rules:
  - host: baga.baga.life # gRPC服务域名，替换为您的域名
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            # gRPC服务
            name: grpc-service
            port:
              number: 50051

3.4、请求方法

在 gRPC 中，方法的全路径（也称为方法描述符）通常被表示为 /。所以，这里的方法和路径可以被描述为：
完整路径：helloworld.Greeter/SayHello
内部请求

grpcurl -plaintext -d '{"name": "World"}' 172.28.69.248:50051 helloworld.Greeter/SayHello

公网请求
![!](https://i-blog.csdnimg.cn/direct/f82a52f97e8146709659bd9b98aa8c2a.png)

测试环境需求完成
ingress-nginx是管理南北向流量很好的一个工具，可以避免在云端频繁对LB进行配置，通过Label也可以实现让ingress-nginx都部署在指定的NodeGroup

一、helm安装ingress-nginx
版本选择，先确认要使用的版本
https://github.com/kubernetes/ingress-nginx

1.1、ingress-nginx vaules文件配置
在这里插入图片描述

kind: DaemonSet
NodeSelector:
nginx: “true”
type: NodePort
nodePorts:
http: “30080”
https: “30443”

创建了pvc存放日志

extraVolumeMounts:
- name: log-volume
mountPath: /var/log/nginx

– Additional volumes to the controller pod.

extraVolumes:
- name: log-volume
persistentVolumeClaim:
claimName: ingress-nginx-pvc
1.2、安装
helm install nginx ingress-nginx/ingress-nginx --namespace ingress-nginx
1.3、给ingress node配置label
NodeSelector:
nginx: “true”
kubectl label node ip-172-28-68-223.ap-southeast-1.compute.internal nginx=true
1.4、获取客户端真实IP
这里配置了forwarded-ip和日志格式

data:
allow-snippet-annotations: “false”
compute-full-forwarded-for: “true”
log-format-upstream: ‘{“@timestamp”:“ $time_iso8601","host":"$ hostname”,“server_ip”:“ $server_addr","client_ip":"$ http_x_forwarded_for”,“xff”:“ $KaTeX parse error: Double subscript at position 7: http_x_̲forwarded_for",\dots$ host”,“url”:“ $uu r i ", " re f erer " : "$ http_referer”,“args”:“ $a r g s ", " u p s t re am t im e " : "$ upstream_response_time”,“responsetime”:“ $request_time","request_method":"$ request_method”,“status”:“ $s t a t u s ", " s i ze " : "$ body_bytes_ss
ent”,“request_length”:“ $request_length","protocol":"$ server_protocol”,“upstreamhost”:“ $upstream_addr","file_dir":"$ request_filename”,“http_user_agent”:“$http_user_agent”}’
use-forwarded-headers: "true
进入容器后查看access.log
真实的客户端IP存储在http_x_forwarded_for字段中

{“@timestamp”:“2024-09-29T07:25:31+00:00”,“host”:“nginx-ingress-nginx-controller-nz9hq”,“server_ip”:“172.28.2.239”,“client_ip”:“xxxxxxxxx, 172.69.63.211”,"
配置ingress资源
我从kubernetes1.18直接升级到了1.29 1.30，ingress-nginx的版本也是大变样，语法也有了一定变化
ingressClassName不在是annotions配置，属于spec资源，另外host和path也有变化，有个模板配置以后问题不大

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: vault-ui
namespace: vault
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
ingressClassName: nginx
rules:

host: vault.baga.life
http:
paths:
- pathType: Prefix
  path: “/”
  backend:
  service:
  name: vault
  port:
  number: 8200
  三、配置grpc ngress资源
  参考文档
  https://help.aliyun.com/zh/ack/ack-managed-and-ack-dedicated/user-guide/use-an-ingress-controller-to-access-grpc-services

3.1、下载grpcurl工具
wget https://github.com/fullstorydev/grpcurl/releases/download/v1.8.7/grpcurl_1.8.7_linux_x86_64.tar.gz
3.2、dp和service配置
grpc-dp.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
name: grpc-service
namespace: baga
spec:
replicas: 1
selector:
matchLabels:
run: grpc-service
template:
metadata:
labels:
run: grpc-service
spec:
containers:
- image: registry.cn-hangzhou.aliyuncs.com/acs-sample/grpc-server:latest
imagePullPolicy: Always
name: grpc-service
ports:
- containerPort: 50051
protocol: TCP
restartPolicy: Always

apiVersion: v1
kind: Service
metadata:
name: grpc-service
namespace: baga
spec:
ports:

port: 50051
protocol: TCP
targetPort: 50051
selector:
run: grpc-service
sessionAffinity: None
type: ClusterIP
3.3、grpc ingress配置
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: grpc-ingress
namespace: baga
annotations:
必须指明后端服务为gRPC服务,核心配置
nginx.ingress.kubernetes.io/backend-protocol: “GRPC”
spec:
ingressClassName: nginx
rules:
host: baga.baga.life # gRPC服务域名，替换为您的域名
http:
paths:
- path: /
  pathType: Prefix
  backend:
  service:
  # gRPC服务
  name: grpc-service
  port:
  number: 50051
  3.4、请求方法
  在 gRPC 中，方法的全路径（也称为方法描述符）通常被表示为 /。所以，这里的方法和路径可以被描述为：
  完整路径：helloworld.Greeter/SayHello
  内部请求