Kubernetes 中完美部署RabbitMQ Cluster

1. rabbitmq-autocluster(已被淘汰)

Kubernetes中可以使用rabbitmq-autocluster这个docker镜像来创建rabbitmq cluster。rabbitmq-autocluster使用autocluster插件去调用kubernetes apiserver获取rabbitmq服务的endpoints获取node节点信息，并自动加入集群。

去docker-hub中看到这货不是官方出的，并且已经3年没更新了。

去github中看到已经被RabbitMQ官方Team接管

结果到rabbitmq官方github中看到也是3年没更新了，并且只支持3.6.x版本。

但是在README中看到，对于3.7.x或更新的版本开始使用 peer discovery subsystem来构建rabbitmq-cluster，并且支持kubernetes。

2. Rabbitmq-peer-discovery-k8s

Github地址??? https://github.com/rabbitmq/rabbitmq-peer-discovery-k8s

参考官方给出的minikube示例即可

2.1 ConfigMap

—

RabbitMQ Config

apiVersion: v1
kind: ConfigMap
metadata:
name: rabbitmq-config
data:
enabled_plugins: |
[rabbitmq_management,rabbitmq_peer_discovery_k8s].
rabbitmq.conf: |
cluster_formation.peer_discovery_backend = rabbit_peer_discovery_k8s
cluster_formation.k8s.host = kubernetes.default.svc.cluster.local
cluster_formation.k8s.address_type = hostname
cluster_formation.node_cleanup.interval = 30
cluster_formation.node_cleanup.only_log_warning = true
cluster_partition_handling = autoheal
queue_master_locator=min-masters
loopback_users.guest = false

2.2 RBAC

—

RabbitMQ ServiceAccount

apiVersion: v1
kind: ServiceAccount
metadata:
name: rabbitmq

RabbitMQ Role

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: endpoint-reader
rules:

• apiGroups: [“”]
  resources: [“endpoints”]
  verbs: [“get”]

---

RabbitMQ RoleBinding

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: endpoint-reader
subjects:

• kind: ServiceAccount
  name: rabbitmq
  roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: endpoint-reader

2.3 Service

—

RabbitMQ Service

apiVersion: v1
kind: Service
metadata:
name: rabbitmq-cluster
labels:
app: rabbitmq-cluster
type: LoadBalancer
spec:
selector:
app: rabbitmq-cluster
ports:
- name: amqp-port
port: 5672
targetPort: 5672
protocol: TCP
- name: mgmt-port
port: 15672
targetPort: 15672
protocol: TCP

RabbitMQ NodePort Service

apiVersion: v1
kind: Service
metadata:
name: rabbitmq-nodeport
labels:
app: rabbitmq-nodeport
type: LoadBalancer
spec:
type: NodePort
selector:
app: rabbitmq-cluster
ports:
- name: amqp-port
nodePort: 30001
port: 5672
targetPort: 5672
protocol: TCP
- name: mgmt-port
nodePort: 30002
port: 15672
targetPort: 15672
protocol: TCP

2.4 StatefulSet

# RabbitMQ-Cluster StatefulSet
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: rabbitmq-cluster
spec:
replicas: 3
selector:
matchLabels:
app: rabbitmq-cluster
serviceName: rabbitmq-internal
template:
metadata:
labels:
app: rabbitmq-cluster
spec:
serviceAccountName: rabbitmq
containers:
- name: rabbitmq
image: rabbitmq:3
livenessProbe:
exec:
# Stage 2 check, more detail at https://www.rabbitmq.com/monitoring.html#health-checks
command: [“rabbitmq-diagnostics”, “status”]
initialDelaySeconds: 60
periodSeconds: 60
timeoutSeconds: 10
readinessProbe:
exec:
# Stage 2 check, more detail at https://www.rabbitmq.com/monitoring.html#health-checks
command: [“rabbitmq-diagnostics”, “status”]
initialDelaySeconds: 60
periodSeconds: 60
timeoutSeconds: 10
ports:
- containerPort: 5672
protocol: TCP
- containerPort: 15672
protocol: TCP
env:
- name: MY_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name # get pod.metadata.name, e.g. rabbitmq-cluster-0
- name: MY_POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace # get pod.metadata.namespace
- name: RABBITMQ_DEFAULT_USER
value: “admin”
- name: RABBITMQ_DEFAULT_PASS
value: “admin”
- name: RABBITMQ_USE_LONGNAME
value: “true”
- name: K8S_SERVICE_NAME
value: “rabbitmq-internal”
- name: RABBITMQ_NODENAME
value: "rabbit@$(MY\_POD\_NAME).$(K8S_SERVICE_NAME).$(MY\_POD\_NAMESPACE).svc.cluster.local"-name:K8S\_HOSTNAME\_SUFFIXvalue:.$(K8S_SERVICE_NAME).$(MY_POD_NAMESPACE).svc.cluster.local
- name: RABBITMQ_ERLANG_COOKIE
value: “SWvCP0Hrqv43NG7GybHC95ntCJKoW8UyNFWnBEWG8TY=” # generator by: echo $(openssl rand -base64 32)
volumeMounts:
- name: config-volume
mountPath: /etc/rabbitmq
volumes:
- name: config-volume
configMap:
name: rabbitmq-config
items:
- key: rabbitmq.conf
path: rabbitmq.conf
- key: enabled_plugins
path: enabled_plugins

3. 运行效果

部署了3个pod

查看Pod日志可以看到，RabbitMQ自动发现节点并加入集群

查看RabbitMQ Cluster状态

root@rabbitmq-cluster-0:/# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbitmq-cluster-0.rabbitmq-internal.demo.svc.cluster.local …
Basics

Cluster name: rabbit@rabbitmq-cluster-0.rabbitmq-internal.demo.svc.cluster.local

Disk Nodes

rabbit@rabbitmq-cluster-0.rabbitmq-internal.demo.svc.cluster.local
rabbit@rabbitmq-cluster-1.rabbitmq-internal.demo.svc.cluster.local
rabbit@rabbitmq-cluster-2.rabbitmq-internal.demo.svc.cluster.local

Running Nodes

rabbit@rabbitmq-cluster-0.rabbitmq-internal.demo.svc.cluster.local
rabbit@rabbitmq-cluster-1.rabbitmq-internal.demo.svc.cluster.local
rabbit@rabbitmq-cluster-2.rabbitmq-internal.demo.svc.cluster.local

Versions

rabbit@rabbitmq-cluster-0.rabbitmq-internal.demo.svc.cluster.local: RabbitMQ 3.8.5 on Erlang 23.0.3
rabbit@rabbitmq-cluster-1.rabbitmq-internal.demo.svc.cluster.local: RabbitMQ 3.8.5 on Erlang 23.0.3
rabbit@rabbitmq-cluster-2.rabbitmq-internal.demo.svc.cluster.local: RabbitMQ 3.8.5 on Erlang 23.0.3

Alarms

(none)

Network Partitions

(none)

Listeners

登录web管理界面查看