记录 K8S 部署 Nacos 集群问题

问题发现：Seata 使用 Nacos 注册时，Nacos 一直返回 server is DOWNnow, detailed error message: Optional[Distro protocol is not initialized]。

Nacos StatefulSet 部署文件

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nacos
  namespace: nacos
spec:
  serviceName: nacos-headless
  replicas: 3
  template:
    metadata:
      labels:
        app: nacos
      annotations:
        pod.alpha.kubernetes.io/initialized: "true"
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: "app"
                    operator: In
                    values:
                      - nacos
              topologyKey: "kubernetes.io/hostname"
      containers:
        - name: nacos
          imagePullPolicy: Always
          image: nacos/nacos-server:latest
          resources:
            requests:
              memory: "2Gi"
              cpu: "500m"
          ports:
            - containerPort: 8848
              name: client
            - containerPort: 9848
              name: client-rpc
            - containerPort: 9849
              name: raft-rpc
            - containerPort: 7848
              name: old-raft-rpc
          env:
            - name: NACOS_REPLICAS
              value: "3"
            - name: MYSQL_SERVICE_HOST
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.host
            - name: MYSQL_SERVICE_DB_NAME
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.db.name
            - name: MYSQL_SERVICE_PORT
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.port
            - name: MYSQL_SERVICE_USER
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.user
            - name: MYSQL_SERVICE_PASSWORD
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.password
            - name: SPRING_DATASOURCE_PLATFORM
              value: "mysql"
            - name: NACOS_SERVER_PORT
              value: "8848"
            - name: NACOS_APPLICATION_PORT
              value: "8848"
            - name: PREFER_HOST_MODE
              value: "hostname"
            - name: NACOS_SERVERS
              value: "nacos-0.nacos-headless.default.svc.cluster.local:8848 nacos-1.nacos-headless.default.svc.cluster.local:8848 nacos-2.nacos-headless.default.svc.cluster.local:8848"
  selector:
    matchLabels:
      app: nacos

根据上述文件启动集群后，Nacos页面访问正常，但 Seata 就是无法注册到 Nacos 上，并且一直报错 server is DOWNnow, detailed error message: Optional[Distro protocol is not initialized]。

进入 Nacos 集群中某一个 Pod 查看日志，发现 Nacos 报错 java.lang.IllegalStateException: unable to find local peer: nacos-2.nacos-headless.default.svc.cluster.local:8848, all peers: []

上述配置文件中修改：value: "nacos-0.nacos-headless.default.svc.cluster.local:8848 nacos-1.nacos-headless.default.svc.cluster.local:8848 nacos-2.nacos-headless.default.svc.cluster.local:8848"

重点：nacos-0.nacos-headless.default.svc.cluster.local:8848 中的 default 对应的是命名空间，因为在上述的配置文件中，我指定了 namespace = nacos，所以这里要改成 nacos。

nacos-0.nacos-headless.nacos.svc.cluster.local:8848

nacos-1.nacos-headless.nacos.svc.cluster.local:8848

nacos-2.nacos-headless.nacos.svc.cluster.local:8848