RocketMQ IPV6集群部署/单机部署

最新推荐文章于 2024-04-21 14:04:42 发布

我叫大王去寻山

最新推荐文章于 2024-04-21 14:04:42 发布

阅读量1.4k

点赞数 1

分类专栏：中间件文章标签： java-rocketmq rocketmq java

本文链接：https://blog.csdn.net/u013539566/article/details/126686145

版权

中间件专栏收录该内容

1 篇文章 0 订阅

订阅专栏

[原创] 转载请注明！

RocketMQ IPV6集群部署/单机部署

IPV6普及程度越来越高，前段时间公司要求全部系统全链路IPV6改造，nginx、keepalived等资料网上还比较多，RockeMQ IPV6改造资料很少，这篇文章主要写部署相关的内容，简单测试的一点代码。

环境

Centos 7.8 [已配置好IPV4/IPV6地址]，如有您服务器未配置IPV6，请自行百度配置好后再来看此文档。
JDK1.8
RocketMQ 4.9.4

问题

RocketMQ官方文档写的4.6.0版本就已经支持IPV6
文档地址：https://rocketmq.apache.org/release_notes/release-notes-4.6.0/

公司之前试用的RocketMQ 4.7.1，配置IPV6后，报错如下：

2022-09-02 18:13:51.388 ERROR 10376 --- [nio-8080-exec-9] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.apache.rocketmq.client.exception.MQClientException: Send [3] times, still failed, cost [32]ms, Topic: TOPIC_1, BrokersSent: [broker-b, broker-a, broker-b]
See http://rocketmq.apache.org/docs/faq/ for further details.] with root cause

org.apache.rocketmq.client.exception.MQBrokerException: CODE: 1  DESC: java.nio.BufferOverflowException, java.base/java.nio.HeapByteBuffer.put(HeapByteBuffer.java:221)
For more information, please visit the url, http://rocketmq.apache.org/docs/faq/
	at org.apache.rocketmq.client.impl.MQClientAPIImpl.processSendResponse(MQClientAPIImpl.java:665) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
	at org.apache.rocketmq.client.impl.MQClientAPIImpl.sendMessageSync(MQClientAPIImpl.java:505) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
	at org.apache.rocketmq.client.impl.MQClientAPIImpl.sendMessage(MQClientAPIImpl.java:487) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
	at org.apache.rocketmq.client.impl.MQClientAPIImpl.sendMessage(MQClientAPIImpl.java:431) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
	at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendKernelImpl(DefaultMQProducerImpl.java:854) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
	at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendDefaultImpl(DefaultMQProducerImpl.java:584) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
	at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1343) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
	at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1289) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
	at org.apache.rocketmq.client.producer.DefaultMQProducer.send(DefaultMQProducer.java:325) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
	at com.sai.rocketmq.demo.service.MqProducerService.send(MqProducerService.java:45) ~[classes!/:1.0.0]
	at com.sai.rocketmq.demo.controller.SuccessController.send(SuccessController.java:30) ~[classes!/:1.0.0]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
	......


2022-09-02 18:14:17.474 ERROR 10376 --- [io-8080-exec-10] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.apache.rocketmq.client.exception.MQBrokerException: CODE: 2  DESC: [REJECTREQUEST]system busy, start flow control for a while
For more information, please visit the url, http://rocketmq.apache.org/docs/faq/] with root cause

org.apache.rocketmq.client.exception.MQBrokerException: CODE: 2  DESC: [REJECTREQUEST]system busy, start flow control for a while
For more information, please visit the url, http://rocketmq.apache.org/docs/faq/
	at org.apache.rocketmq.client.impl.MQClientAPIImpl.processSendResponse(MQClientAPIImpl.java:665) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
	at org.apache.rocketmq.client.impl.MQClientAPIImpl.sendMessageSync(MQClientAPIImpl.java:505) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
	at org.apache.rocketmq.client.impl.MQClientAPIImpl.sendMessage(MQClientAPIImpl.java:487) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
	at org.apache.rocketmq.client.impl.MQClientAPIImpl.sendMessage(MQClientAPIImpl.java:431) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
	at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendKernelImpl(DefaultMQProducerImpl.java:854) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
	at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendDefaultImpl(DefaultMQProducerImpl.java:584) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
	at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1343) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
	at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1289) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
	at org.apache.rocketmq.client.producer.DefaultMQProducer.send(DefaultMQProducer.java:325) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
	at com.sai.rocketmq.demo.service.MqProducerService.send(MqProducerService.java:45) ~[classes!/:1.0.0]
	at com.sai.rocketmq.demo.controller.SuccessController.send(SuccessController.java:30) ~[classes!/:1.0.0]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[spring-web-5.3.13.jar!/:5.3.13]
	......

解决方案：升级RocketMQ 4.9.4版本，配置IPV6正常使用。

集群部署RocketMQ

双主双备

hostname	IPV6	主broker	备broker
mq1	2409:8a3c:4d9:6001::1001	broker-a	broker-b-s
mq2	2409:8a3c:4d9:6001::1002	broker-b	broker-a-s

1. 下载安装包

官网：https://rocketmq.apache.org/dowloading/releases/

这里我下载的版本是：Binary: rocketmq-all-4.9.4-bin-release.zip [PGP] [SHA512]

2. 解压

unzip rocketmq-all-4.9.4-bin-release -d /home/xx

3. 修改/覆盖配置文件 [同步复制策略|异步刷盘策略]

# PS：复制策略和刷盘策略可根据需求自行修改

# -----------------------  mq1机器  ------------------------------

# 3.1 mq1机器[2409:8a3c:4d9:6001::1001] nameserver配置 conf/2m-2s-async/namesrv.properties
listenPort=12011

# 3.2 mq1机器[2409:8a3c:4d9:6001::1001] 主broker配置 conf/2m-2s-async/broker-a.properties
brokerClusterName=rocketmq-cluster
brokerName=broker-a
brokerId=0
# nameServer地址，分号分割
namesrvAddr=[2409:8a3c:4d9:6001::1001]:12011;[2409:8a3c:4d9:6001::1002]:12011
listenPort=12012
haListenPort=12013
deleteWhen=04
fileReservedTime=48
storePathRootDir=/home/xx/rocketmq/store-m
storePathCommitLog=/home/xx/rocketmq/store-m/commitlog
storePathConsumeQueue=/home/xx/rocketmq/store-m/consumequeue
storePathIndex=/home/xx/rocketmq/store-m/index
storeCheckpoint=/home/xx/rocketmq/store-m/checkpoint
abortFile=/home/xx/rocketmq/store-m/abort
brokerRole=SYNC_MASTER
flushDiskType=ASYNC_FLUSH
# 强制指定本机IP，需要根据每台机器进行修改。官方介绍可为空，系统默认自动识别，但多网卡时IP地址可能读取错误
brokerIP1=2409:8a3c:4d9:6001::1001

# 3.3 mq1机器[2409:8a3c:4d9:6001::1001] 备broker配置 conf/2m-2s-async/broker-b-s.properties
brokerClusterName=rocketmq-cluster
brokerName=broker-b
brokerId=1
#nameServer地址，分号分割
namesrvAddr=[2409:8a3c:4d9:6001::1001]:12011;[2409:8a3c:4d9:6001::1002]:12011
listenPort=12022
haListenPort=12023
deleteWhen=04
fileReservedTime=48
storePathRootDir=/home/xx/rocketmq/store-s
storePathCommitLog=/home/xx/rocketmq/store-s/commitlog
storePathConsumeQueue=/home/xx/rocketmq/store-s/consumequeue
storePathIndex=/home/xx/rocketmq/store-s/index
storeCheckpoint=/home/xx/rocketmq/store-s/checkpoint
abortFile=/home/xx/rocketmq/store-s/abort
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH
#强制指定本机IP，需要根据每台机器进行修改。官方介绍可为空，系统默认自动识别，但多网卡时IP地址可能读取错误
brokerIP1=2409:8a3c:4d9:6001::1001

# ----------------------  mq2机器  -------------------------------

# 3.4 mq2机器[2409:8a3c:4d9:6001::1002] nameserver配置 conf/2m-2s-async/namesrv.properties
listenPort=12011

# 3.5 mq2机器[2409:8a3c:4d9:6001::1002] 主broker配置 conf/2m-2s-async/broker-b.properties
brokerClusterName=rocketmq-cluster
brokerName=broker-b
brokerId=0
#nameServer地址，分号分割
namesrvAddr=[2409:8a3c:4d9:6001::1001]:12011;[2409:8a3c:4d9:6001::1002]:12011
listenPort=12012
haListenPort=12013
deleteWhen=04
fileReservedTime=48
storePathRootDir=/home/xx/rocketmq/store-m
storePathCommitLog=/home/xx/rocketmq/store-m/commitlog
storePathConsumeQueue=/home/xx/rocketmq/store-m/consumequeue
storePathIndex=/home/xx/rocketmq/store-m/index
storeCheckpoint=/home/xx/rocketmq/store-m/checkpoint
abortFile=/home/xx/rocketmq/store-m/abort
brokerRole=SYNC_MASTER
flushDiskType=ASYNC_FLUSH
#多网卡可以强制指定本机IP，需要根据每台机器进行修改。为空，系统默认自动识别
brokerIP1=2409:8a3c:4d9:6001::1002

# 3.6 mq2机器[2409:8a3c:4d9:6001::1002] 备broker配置 conf/2m-2s-async/broker-a-s.properties
brokerClusterName=rocketmq-cluster
brokerName=broker-a
brokerId=1
#nameServer地址，分号分割
namesrvAddr=[2409:8a3c:4d9:6001::1001]:12011;[2409:8a3c:4d9:6001::1002]:12011
listenPort=12022
haListenPort=12023
deleteWhen=04
fileReservedTime=48
storePathRootDir=/home/np/rocketmq/store-s
storePathCommitLog=/home/np/rocketmq/store-s/commitlog
storePathConsumeQueue=/home/np/rocketmq/store-s/consumequeue
storePathIndex=/home/np/rocketmq/store-s/index
storeCheckpoint=/home/np/rocketmq/store-s/checkpoint
abortFile=/home/np/rocketmq/store-s/abort
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH
#多网卡可以强制指定本机IP，需要根据每台机器进行修改。为空，系统默认自动识别
brokerIP1=2409:8a3c:4d9:6001::1002

4. 启动/停止服务

PS 切记！禁止直接kill broker进程！如果想停止mq服务，可以使用命令停止！

# 启动：先启动nameserver集群，再启动master-broker，最后启动slave-broker，具体操作如下：
# 1. 启动mq1和mq2两台虚拟机的 nameserver 服务
# 进入到rocketmq的bin目录
> sh mqnamesrv -c conf/2m-2s-async/namesrv.properties

# 2. 启动mq1和mq2两台虚拟机的 主broker 服务
# 2.1 启动mq1 主broker
> sh mqbroker -c conf/2m-2s-async/broker-a.properties
# 2.2 启动mq2 主broker
> sh mqbroker -c conf/2m-2s-async/broker-b.properties

# 3. 启动mq1和mq2两台虚拟机的 备broker 服务
# 2.1 启动mq1 备broker
> sh mqbroker -c conf/2m-2s-async/broker-b-s.properties
# 2.2 启动mq2 备broker
> sh mqbroker -c conf/2m-2s-async/broker-a-s.properties

# 停止服务，先停 broker服务，再停 nameserver服务
> sh mqshutdown broker
> sh mqshutdown namesrv

5. 代码测试

POM文件引入：

<dependency>
	<groupId>org.apache.rocketmq</groupId>
	<artifactId>rocketmq-client</artifactId>
	<version>4.9.4</version>
</dependency>

5.1 生产消息

    @SneakyThrows
    public static void main(String[] args) {
        DefaultMQProducer producer = new DefaultMQProducer("PRODUCER_TOPIC_1_GROUP");
        producer.setNamesrvAddr("[2409:8a3c:4d9:6001::1001]:12011;[2409:8a3c:4d9:6001::1002]:12011");
        producer.start();
        Message msg = new Message("TOPIC_1", "Hello RocketMQ!".getBytes());
        SendResult result = producer.send(msg);
        System.out.println("result = " + result);
        TimeUnit.SECONDS.sleep(1);
        producer.shutdown();
    }

5.2 消费消息

    @SneakyThrows
    public static void main(String[] args) {
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("CONSUMER_TOPIC_1_GROUP");
        consumer.setNamesrvAddr("[2409:8a3c:4d9:6001::1001]:12011;[2409:8a3c:4d9:6001::1002]:12011");
        consumer.subscribe("TOPIC_1", "*");
        consumer.registerMessageListener((MessageListenerConcurrently) (msgs, consumeConcurrentlyContext) -> {
            for (MessageExt msg : msgs) {
                String str = new String(msg.getBody());
                System.out.println(String.format("WANGZZ >>>>>> MSGID:%s | body: %s", msg.getMsgId(), str));
            }
            return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
        });
        consumer.start();
    }

我叫大王去寻山

关注

1
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
RocketMQ IPV6集群部署/单机部署

IPV6普及程度越来越高，前段时间公司要求全部系统全链路IPV6改造，nginx、keepalived等资料网上还比较多，RockeMQ IPV6改造资料很少，这篇文章主要写部署相关的内容，简单测试的一点代码。
复制链接

扫一扫