上一篇我们介绍了RocketMQ中的基础知识和环境搭建,在本篇中我们将继续介绍如何通过Java代码去操作RocketMQ。
RocketMQ-生产者使用
- 创建生产者对象DefaultMQProducer,生产者组的名称应当唯一
- 设置NamesrvAddr
- 启动生产者服务
- 创建消息并发送
编写简单的消息生产者类,如下所示。
public class Producer {
public static void main(String[] args) throws MQClientException, RemotingException, InterruptedException, MQBrokerException {
// InterruptedException 多线程打断的时候,会抛出这个异常
DefaultMQProducer defaultMQProducer = new DefaultMQProducer("test_quick_producer_name");
defaultMQProducer.setNamesrvAddr(NAMESRV_ADDR);
defaultMQProducer.start();
// 主题、标签(主要用来做消息过滤)、用户自定义的key(唯一的标识)、消息内容实体(byte类型)
for (int i = 0; i < 5; i++) {
// 创建消息
// Topic 与 MessageQueue为一个一对多的关系
Message message = new Message("test_quick_start_topic", "TagA", "Key" + i, ("Hello rocketMQ : " + i).getBytes());
// 发送消息
SendResult sendResult = defaultMQProducer.send(message, 1000 * 1000);
System.out.println("消息发出,得到返回值,sendResult : " + JSON.toJSONString(sendResult));
}
defaultMQProducer.shutdown();
}
}
执行上述生产者类后,可以发现控制台输出了以下内容。由messageQueue中的queueId可以发现该Topic具有4个messageQueue,Id分别为0、1、2、3。由此可知Topic与MessageQueue的关系为一对多关系,Topic的MessageQueue数量可以通过broker配置文件中的defaultTopicQueueNums来进行配置。
消息发出,得到返回值,sendResult : {"messageQueue":{"brokerName":"broker-a","queueId":3,"topic":"test_quick_start_topic"},"msgId":"0A0239B2341718B4AAC22642202E0000","offsetMsgId":"2F6B5A2400002A9F0000000000000000","queueOffset":0,"regionId":"DefaultRegion","sendStatus":"SEND_OK","traceOn":true}
消息发出,得到返回值,sendResult : {"messageQueue":{"brokerName":"broker-a","queueId":0,"topic":"test_quick_start_topic"},"msgId":"0A0239B2341718B4AAC2264220F50001","offsetMsgId":"2F6B5A2400002A9F00000000000000CB","queueOffset":0,"regionId":"DefaultRegion","sendStatus":"SEND_OK","traceOn":true}
消息发出,得到返回值,sendResult : {"messageQueue":{"brokerName":"broker-a","queueId":1,"topic":"test_quick_start_topic"},"msgId":"0A0239B2341718B4AAC22642214F0002","offsetMsgId":"2F6B5A2400002A9F0000000000000196","queueOffset":0,"regionId":"DefaultRegion","sendStatus":"SEND_OK","traceOn":true}
消息发出,得到返回值,sendResult : {"messageQueue":{"brokerName":"broker-a","queueId":2,"topic":"test_quick_start_topic"},"msgId":"0A0239B2341718B4AAC2264221CC0003","offsetMsgId":"2F6B5A2400002A9F0000000000000261","queueOffset":0,"regionId":"DefaultRegion","sendStatus":"SEND_OK","traceOn":true}
消息发出,得到返回值,sendResult : {"messageQueue":{"brokerName":"broker-a","queueId":3,"topic":"test_quick_start_topic"},"msgId":"0A0239B2341718B4AAC2264221FB0004","offsetMsgId":"2F6B5A2400002A9F000000000000032C","queueOffset":1,"regionId":"DefaultRegion","sendStatus":"SEND_OK","traceOn":true}
10:17:48.429 [NettyClientSelector_1] INFO RocketmqRemoting - closeChannel: close the connection to remote address[47.107.90.36:10911] result: true
10:17:48.441 [NettyClientSelector_1] INFO RocketmqRemoting - closeChannel: close the connection to remote address[47.107.90.36:10909] result: true
10:17:48.441 [NettyClientSelector_1] INFO RocketmqRemoting - closeChannel: close the connection to remote address[47.107.90.36:9876] result: true
RocketMQ-消费者使用
- 创建消费者对象DefaultMQPushConsumer;
- 设置NamesrvAddr及其消费位置ConsumerFromWhere(用于设置消费消息的起始点);第一次启动时会从指定的消费位置ConsumerFromWhere去消费。随着消息的消费,这个位置会保存在server和broker中,不在为指定的ConsumerFromWhere。
- 进行订阅主题subscribe;
- 注册监听并消费registerMessageListener
编写简单的消费者类,代码如下。
public class Consumer {
public static void main(String[] args) throws MQClientException {
DefaultMQPushConsumer defaultMQPushConsumer = new DefaultMQPushConsumer("test_quick_consumer_name");
defaultMQPushConsumer.setNamesrvAddr(NAMESRV_ADDR);
// 设置从最末尾开始消费
defaultMQPushConsumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_LAST_OFFSET);
// 订阅一个topic,并区分tag,使用*则是模糊匹配
defaultMQPushConsumer.subscribe("test_quick_start_topic", "*");
// 注册监听并消费registerMessageListener
defaultMQPushConsumer.registerMessageListener(new MessageListenerConcurrently() {
@Override
public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> messageExtList, ConsumeConcurrentlyContext context) {
MessageExt messageExt = messageExtList.get(0);
try {
String topic = messageExt.getTopic();
String msgId = messageExt.getMsgId();
String keys = messageExt.getKeys();
if ("Key1".equals(keys)) {
System.err.println("发送消息失败");
int a = 1 / 0;
}
byte[] body = messageExt.getBody();
String message = new String(body, RemotingHelper.DEFAULT_CHARSET);
System.err.println("接收到消息:topic=" + topic + ",msgId=" + msgId + ",tags=" + keys + ",body=" + message);
} catch (Exception e) {
e.printStackTrace();
// 后面去重试消费。messageDelayLevel=1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h
// 底层是维护了一个定时任务
int reconsumeTimes = messageExt.getReconsumeTimes();
System.out.println("reconsumeTimes:" + reconsumeTimes);
if (reconsumeTimes == 3) {
// 记录日志,后续做补偿处理
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
}
return ConsumeConcurrentlyStatus.RECONSUME_LATER;
}
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
}
});
// 启动消费者
defaultMQPushConsumer.start();
System.err.println("消费者端启动");
}
}
执行上述代码,控制台输出如下所示。可以发现在发生异常后,RocketMQ自动进行了消息的重发尝试。我们在broker的启动日志中,可以发现messageDelayLevel的配置,其实就是配置了RocketMQ消息消费失败后,重试等待的时间。
消费者端启动
接收到消息:topic=test_quick_start_topic,msgId=0A0239B23F6818B4AAC22770C6990000,tags=Key0,body=Hello rocketMQ : 0
发送消息失败
接收到消息:topic=test_quick_start_topic,msgId=0A0239B23F6818B4AAC22770C7B40002,tags=Key2,body=Hello rocketMQ : 2
接收到消息:topic=test_quick_start_topic,msgId=0A0239B23F6818B4AAC22770C80F0003,tags=Key3,body=Hello rocketMQ : 3
接收到消息:topic=test_quick_start_topic,msgId=0A0239B23F6818B4AAC22770C89C0004,tags=Key4,body=Hello rocketMQ : 4
java.lang.ArithmeticException: / by zero
at com.xinghaol.rocketmq.quickstart.Consumer$1.consumeMessage(Consumer.java:42)
at org.apache.rocketmq.client.impl.consumer.ConsumeMessageConcurrentlyService$ConsumeRequest.run(ConsumeMessageConcurrentlyService.java:417)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
reconsumeTimes:0
发送消息失败
java.lang.ArithmeticException: / by zero
at com.xinghaol.rocketmq.quickstart.Consumer$1.consumeMessage(Consumer.java:42)
at org.apache.rocketmq.client.impl.consumer.ConsumeMessageConcurrentlyService$ConsumeRequest.run(ConsumeMessageConcurrentlyService.java:417)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
reconsumeTimes:1
发送消息失败
java.lang.ArithmeticException: / by zero
at com.xinghaol.rocketmq.quickstart.Consumer$1.consumeMessage(Consumer.java:42)
at org.apache.rocketmq.client.impl.consumer.ConsumeMessageConcurrentlyService$ConsumeRequest.run(ConsumeMessageConcurrentlyService.java:417)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
reconsumeTimes:2
RocketMQ四种集群
- 单点模式
- 主从模式,较高可用
- 双主模式,无从节点
- 双主双从模式,多主多从模式,在实际生产环境中会较多的使用。
RocketMQ-主从模式
主从模式可以保证消息的及时性和可靠性。投递一条消息后,关闭了主节点;从节点可以继续向消费者提供数据进行消费,但是无法接受消息。此时主节点上线后,需要和从节点同步offset消费进度。