介绍
基于Erlang开发消息队列(AMQP)
在分布式系统中, 有一些功能我们希望能够提高系统稳定性, 比如说支付、订单功能, 服务后移, 长时间操作的功能, 同步数据
我们通过监听数据变化实现功能联动
![9c2622a2bc772a17658091103c6ea376.png](https://i-blog.csdnimg.cn/blog_migrate/5131fcf67ddbdd7518f085f99facd03a.jpeg)
特点
分布式
各节点互相冗余元数据(erlang.cookie , 队列、交换机、绑定元数据、vhost元数据)实现分布式集群(类似于session), 通过转发达到集群任意节点进入都是集群主节点
可以任意增加节点实现集群水平扩容,但是如果集群是多主节点是不能任意删除节点, 因为实际数据还是存在于各个主节点中, 但是我们可以使用镜像策略来解决这个问题
# 注意使用正则表达式匹配vhost目录
$ rabbitmqctl set_policy ha-all "^ha." '{"ha-mode":"all"}'
可持久化
// 投递消息时设置持久化
channel.queueDeclare("test", true, false, false, null)
// 投递模式设置为2
channel.basicPublish(x, x, MessageProperties.PERSISTENT_TEXT_PLAIN,x)
原理
发送消息: Client -> Exchange -Binding-> Queue -> Server
接收消息: Server -> Channe(线程池) -> Client
Binding可用模式为:
1、Direct, 完全匹配绑定的Queue的名称
2、fanout, 所有Queue
3、topic, 适用于通配符匹配的Queue
Message由Header(属性集合)和Body组成
可以通过Vhost去实现多租户(权限)
客户端可以通过设置AutoACK=false去手动应答以实现更加稳定的数据处理请求, 例如推送到死信队列
对比
相比于Kafka, RabbitMQ更加稳定, 适用于复杂业务场景而非数据传递
相比于ActiveMQ,RabbitMQ更高性能更丰富, 更适用于云原生
相比于RocketMQ, 更加国际化, 更加方便
在K8S中搭建RabbitMQ
参考 https://github.com/rabbitmq/rabbitmq-peer-discovery-k8s
核心基于Kubernetes Discovery 插件(rabbitmq_peer_discovery_k8s)
在使用 StatefulSet 时使用 主机 访问即可
Namespace
apiVersion: v1
kind: Namespace
metadata:
name: middleware
labels:
name: middleware
ConfigMap
---
apiVersion: v1
kind: ConfigMap
metadata:
name: rabbitmq-config
namespace: middleware
data:
enabled_plugins: |
[rabbitmq_management,rabbitmq_peer_discovery_k8s,rabbitmq_shovel,rabbitmq_shovel_management,accept,prometheus,prometheus_httpd,prometheus_rabbitmq_exporter,prometheus_process_collector].
rabbitmq.conf: |
cluster_formation.peer_discovery_backend = rabbit_peer_discovery_k8s
cluster_formation.k8s.host = kubernetes.default.svc.cluster.local
cluster_formation.k8s.address_type = hostname
cluster_formation.node_cleanup.interval = 10
cluster_formation.node_cleanup.only_log_warning = true
cluster_partition_handling = autoheal
queue_master_locator=min-masters
loopback_users.guest = false
cluster_formation.randomized_startup_delay_range.min = 0
cluster_formation.randomized_startup_delay_range.max = 2
cluster_formation.k8s.service_name = rabbitmq-headless
cluster_formation.k8s.hostname_suffix = .rabbitmq-headless.rabbitmq.svc.cluster.local
vm_memory_high_watermark.absolute = 3.2GB
disk_free_limit.absolute = 10GB
default_user = test
default_pass = test
配置参数讲解:
| 配置 | 含义 | | :----------------------------------------------------------- | :----------------------------------------------------------: | | cluster_formation.peer_discovery_backend = rabbit_peer_discovery_k8s | 指定集群节点发现机制 | | cluster_formation.k8s.host = kubernetes.default.svc.cluster.local | Kubernetes API 主机名 | | cluster_formation.k8s.address_type = hostname | 访问其他节点时使用的方式 | | cluster_formation.node_cleanup.interval = 10 | 每隔多久清理上次获取的节点主机并获取新的节点主机 | | cluster_formation.node_cleanup.only_log_warning = true | 记录日志 | | cluster_partition_handling = autoheal | 网络异常时, 使用网络分区将异常的节点排离在分区之外, 这里设置为自动决定一个获胜的(winning)分区,然后重启不在这个分区中的节点以恢复网络分区。一个获胜的分区是指客户端连接最多的一个分区。如果产生一个平局,既有两个或者多个分区的客户端连接数一样多,那么节点数最多的一个分区就是获胜的分区。如果此时节点数也一样多,将会以一种特殊的方式来挑选获胜分区 | | queue_master_locator=min-masters | 队列主节点的策略为选择托管最少数量的绑定主节点的节点 | | cluster_formation.randomized_startup_delay_range.min = 0 cluster_formation.randomized_startup_delay_range.max = 2 | 随机延迟初始化时间, 单位为秒 | | cluster_formation.k8s.service_name = rabbitmq-headless | 对应部署对象的K8S_SERVICE_NAME变量 | | cluster_formation.k8s.hostname_suffix = .rabbitmq-headless.rabbitmq.svc.cluster.local | 集群访问的后缀 | | vm_memory_high_watermark.absolute = 3.2GB | 内存上限 | | disk_free_limit.absolute = 10GB | 磁盘上限 | | default_user = test default_pass = test | 默认初始的账号及其密码 | | | |
Service
---
kind: Service
apiVersion: v1
metadata:
name: rabbitmq-headless
namespace: middleware
spec:
clusterIP: None
publishNotReadyAddresses: true
ports:
- name: amqp
port: 5672
- name: http
port: 15672
selector:
app: rabbitmq
---
kind: Service
apiVersion: v1
metadata:
namespace: middleware
name: rabbitmq-service
spec:
type: NodePort
ports:
- name: http
protocol: TCP
port: 15672
targetPort: 15672
- name: amqp
protocol: TCP
port: 5672
selector:
app: rabbitmq
RBAC
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: rabbitmq
namespace: middleware
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: endpoint-reader
namespace: middleware
rules:
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: endpoint-reader
namespace: middleware
subjects:
- kind: ServiceAccount
name: rabbitmq
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: endpoint-reader
StatefulSet
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: rabbitmq
namespace: middleware
spec:
serviceName: rabbitmq-headless
selector:
matchLabels:
app: rabbitmq
replicas: 1
template:
metadata:
labels:
app: rabbitmq
annotations:
scheduler.alpha.kubernetes.io/affinity: >
{
"podAntiAffinity": {
"requiredDuringSchedulingIgnoredDuringExecution": [{
"labelSelector": {
"matchExpressions": [{
"key": "app",
"operator": "In",
"values": ["rabbitmq"]
}]
},
"topologyKey": "kubernetes.io/hostname"
}]
}
}
spec:
nodeSelector:
middleware: "middleware"
serviceAccountName: rabbitmq
terminationGracePeriodSeconds: 10
containers:
- name: rabbitmq
image: registry-vpc.cn-shenzhen.aliyuncs.com/heygears/rabbitmq:3.7
resources:
limits:
cpu: 0.5
memory: 2Gi
requests:
cpu: 0.3
memory: 2Gi
volumeMounts:
- name: config-volume
mountPath: /etc/rabbitmq
readOnly: true
- name: rabbitmq-data
mountPath: /var/lib/rabbitmq/mnesia
subPath: rabbitmq-ha
ports:
- name: http
protocol: TCP
containerPort: 15672
- name: amqp
protocol: TCP
containerPort: 5672
livenessProbe:
exec:
command: ["rabbitmqctl", "status"]
initialDelaySeconds: 60
periodSeconds: 60
timeoutSeconds: 5
readinessProbe:
exec:
command: ["rabbitmqctl", "status"]
initialDelaySeconds: 20
periodSeconds: 60
timeoutSeconds: 5
imagePullPolicy: Always
env:
- name: HOSTNAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: RABBITMQ_USE_LONGNAME
value: "true"
- name: RABBITMQ_NODENAME
value: "rabbit@$(HOSTNAME).rabbitmq-headless.middleware.svc.cluster.local"
- name: RABBITMQ_ERLANG_COOKIE
value: "mycookie"
volumes:
- name: config-volume
configMap:
name: rabbitmq-config
items:
- key: rabbitmq.conf
path: rabbitmq.conf
- key: enabled_plugins
path: enabled_plugins
- name: rabbitmq-data
persistentVolumeClaim:
claimName: nfs-pvc-middleware
PV/PVC
---
apiVersion: v1
kind: PersistentVolume
metadata:
namespace: middleware
name: nfs-pv-middleware
labels:
app: nfs
spec:
capacity:
storage: 500Gi
accessModes:
- ReadWriteMany
mountOptions:
- vers=3
- nolock,tcp,noresvport
nfs:
path: /
server: xxx.nas.aliyuncs.com
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
namespace: middleware
name: nfs-pvc-middleware
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 100Gi
selector:
matchLabels:
app: nfs
IngressRoute
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: rabbitmq-ingress-route
namespace: middleware
spec:
entryPoints:
- web
routes:
- match: Host(`xxx.com`) && PathPrefix(`/`)
kind: Rule
services:
- name: rabbitmq-service
port: 15672