记一次istio访问rabbitmq有问题的处理经历

问题描述

最近在istio里部署好rabbitmq后，可以访问管理端口15672, 但是5672的amqp端口，java的spring-boot程序去连，一直报如下错误

2024-05-30 10:07:06.557  WARN 1 --- [.168.3.124:5672] c.r.c.impl.ForgivingExceptionHandler     : An unexpected connection driver error occurred (Exception message: Connection reset)
2024-05-30 10:07:11.560  WARN 1 --- [.168.3.124:5672] c.r.c.impl.ForgivingExceptionHandler     : An unexpected connection driver error occurred (Exception message: Connection reset)

上网查了资料后，有说要加新端口4369才可。但是加了，还是不行。经过和同事研究，一番折腾后，发现有两种解决方案。

解决方案1

将rabbitmq去除边车模式，首先将所在的namespace去掉istio-injection, 即不受istio管理，然后只需配deployment和service服务即可。这时通过k get pod查看mq的pod的ready, 应该是1/1。然后java程序连rabbimq OK了。下面是相关指令：

# 去除namespace的istio-injection, sup是namespace的名称
kubectl label namespace sup istio-injection-
# 或将其disable也可
kubectl label --overwrite namespace sup istio-injection=disabled

# 然后可用如下指令检查执行情况
kubectl get namespace -L istio-injection

补充一下，除通过namespace去除边车外，还可单独针对某个deployment(即pod)来去除边车，具体配置举例如下

apiVersion: apps/v1
kind: Deployment
metadata:
  name: consent-login
spec:
  selector:
    matchLabels:
      app: consent-login
  replicas: 1
  template:
    metadata:
      labels:
        app: consent-login
      annotations:
        sidecar.istio.io/inject: "false"  # 这就是关键所在

解决方案2

难道rabbitmq就不能在边车里使用么？经过再检查，发现是destinationrule和virtualservice配置里有版本控制，而deployment那却没有，导致不匹配。将版本去掉，即使将rabbitmq里放边车里，也可以正常访问了。

版本说明

k8s : v1.26.10
istio: v1.21.0

配置文件清单

文件清单：

./
├── base
│   ├── deployment.yaml
│   ├── destinationrule.yaml
│   ├── kustomization.yaml
│   ├── service.yaml
│   └── virtualservice.yaml
└── dev
    ├── deployment.yaml
    ├── kustomization.yaml
    └── mq.env

base/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sup-mq
spec:
  selector:
    matchLabels:
      app: sup-mq
  replicas: 1
  template:
    metadata:
      labels:
        app: sup-mq
    spec:
      containers:
        - name: sup-mq
          image: rabbitmq:3.9.10-management
          ports:
          - containerPort: 5672
          - containerPort: 15672

base/destinationrule.yaml
( 原来就是下面加的版本v1在deployment里没有对应导致有问题，去掉就好了。 如要保留，那么在deployment那也要加相应的版本才行)

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: sup-mq
spec:
  host: sup-mq
  #subsets:
  #  - name: v1
  #    labels:
  #      version: v1

base/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - deployment.yaml
  - service.yaml
  - virtualservice.yaml
  - destinationrule.yaml

base/service.yaml

kind: Service
apiVersion: v1
metadata:
  name: sup-mq
spec:
  selector:
    app: sup-mq
  ports:
    - protocol: TCP
      name: http-admin
      port: 80
      targetPort: 15672
    - protocol: TCP
      name: amqp
      port: 5672
      targetPort: 5672

base/virtualservice.yaml
（版本不要有，如有，deployment那也需要）

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: sup-mq
spec:
  hosts:
  - sup-mq
  tcp:
  - match:
    - port: 80
    route:
    - destination:
        host: sup-mq
        #subset: v1
        port:
          number: 15672
  - match:
    - port: 5672
    route:
    - destination:
        host: sup-mq
        #subset: v1
        port:
          number: 5672

dev/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sup-mq
spec:
  selector:
    matchLabels:
      app: sup-mq
  replicas: 1
  template:
    metadata:
      labels:
        app: sup-mq
    spec:
      containers:
        - name: sup-mq
          envFrom:
            - secretRef:
                name: mq-env

dev/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: sup

resources:
  - ../base

patchesStrategicMerge:
- deployment.yaml

secretGenerator:
  - name: mq-env
    envs:
      - mq.env
    type: Opaque

dev/mq.env

RABBITMQ_DEFAULT_USER=admin
RABBITMQ_DEFAULT_PASS=xxxxxx
TZ=Asia/Shanghai