消息队列入门一之helloword

于 2019-02-15 11:14:00 发布
阅读量148 收藏
点赞数
文章标签： git python java
原文链接：https://my.oschina.net/xlpapapa/blog/3010374
版权

2019独角兽企业重金招聘Python工程师标准>>> hot3.png

什么是MQ

消息队列是为了让系统解耦，将不是主要流程的操作异步调用、流量削峰。 （经典例子：下单成功，发送短信和发送邮箱提醒可以交给其他线程处理，但是分布式情况下，下单流程把这些操作丢给MQ就不用管了和ajax一个道理，让短信系统和邮箱系统 轮询拿属于自己的任务。）

常见MQ

常见的有activeMQ、rocketMQ、kafka。 多个系统需要一个系统的接口时，如果按照正常的方式就是死循环轮训去访问接口，那么耦合性就非常大，服务器压力也很大。如果信息提供方自己把共享的信息推到一个服务器上就不管，消费者也是轮巡拿数据就好了。所以消息队列也叫做消息中间间。

怎么入门

入门都要helloworld，MQ（message queue），也不例外，实现的思想按道理就是先启动MQ的服务，生产者系统（根据ip、端口和一些规则）推信息到MQ，消费者（根据规则）拿信息，MQ也有自己的控制台要能看到谁在这里放了什么数据，谁拿了数据。以便管理。

入门之常见MQ-helloworld

这里用阿里的rocketMQ做例子（现在已经给apache开源了）

1. 启动RocketMQ

官方文档 http://rocketmq.apache.org/docs/quick-start 启动要求：

下他个二进制版本 http://rocketmq.apache.org/dowloading/releases/

nohup和&后台运行，进程查看及终止,比如nohup ..... & 启动 Now execute the following commands to unpack 4.4.0 source release and build the binary artifact.(如果下的不是二进制版本就要执行)

  unzip rocketmq-all-4.4.0-source-release.zip
  cd rocketmq-all-4.4.0/
  mvn -Prelease-all -DskipTests clean install -U
  cd distribution/target/apache-rocketmq

Start Name Server

  > nohup sh bin/mqnamesrv &
  > tail -f ~/logs/rocketmqlogs/namesrv.log
  The Name Server boot success...

Start Broker

  nohup sh bin/mqbroker -n localhost:9876 &
  tail -f ~/logs/rocketmqlogs/broker.log 
  The broker[%s, 172.30.30.233:10911] boot success...

如果不想用这种方法，也可以开两个窗口分别启动这两个服务。更直观 期间就遇到了JAVA_HOME没配置启动不了的问题

2. 编写生产者和消费者的代码

  1. 新建maven工程 pom.xml:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>seniordemo</groupId>
  <artifactId>gzydemo</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>gzydemo</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <dependencies>
  <!-- https://mvnrepository.com/artifact/com.alibaba/fastjson -->
<dependency>
    <groupId>com.alibaba</groupId>
    <artifactId>fastjson</artifactId>
    <version>1.2.47</version>
</dependency>
  <!-- https://mvnrepository.com/artifact/io.netty/netty-all -->
<dependency>
    <groupId>io.netty</groupId>
    <artifactId>netty-all</artifactId>
    <version>4.1.32.Final</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.rocketmq/rocketmq-client -->
<dependency>
    <groupId>org.apache.rocketmq</groupId>
    <artifactId>rocketmq-client</artifactId>
    <version>4.4.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.rocketmq/rocketmq-common -->
<dependency>
    <groupId>org.apache.rocketmq</groupId>
    <artifactId>rocketmq-common</artifactId>
    <version>4.4.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-api -->
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>1.7.25</version>
</dependency>
 <dependency>
       <groupId>org.slf4j</groupId>
       <artifactId>slf4j-nop</artifactId>
       <version>1.7.2</version>
   </dependency>
  
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
</project>

2、java代码

package seniordemo.gzydemo;

import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.client.producer.SendResult;
import org.apache.rocketmq.common.message.Message;

public class Publisher {
	public static void main(String[] args) throws Exception {
		//传入producer group
		DefaultMQProducer producer = new DefaultMQProducer("gzy_producer");
		producer.setNamesrvAddr("localhost:9876");
		producer.setRetryTimesWhenSendFailed(10);
		producer.start();
		for (int i = 0; i < 20; i++) {
			Message msg = new Message("TopicQuickStart", "gzy_tag", ("发射动感光波" + i).getBytes());
			try {
				SendResult sendResult = producer.send(msg, 2000);// 2秒钟内未发送成功自动重试
				System.out.println(sendResult);
			} catch (Exception e) {
				e.printStackTrace();
				Thread.sleep(1000);
			}
		}

		producer.shutdown();

	}
}


package seniordemo.gzydemo;

import java.util.List;

import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.common.consumer.ConsumeFromWhere;
import org.apache.rocketmq.common.message.MessageExt;

public class Reciver  {
	public static void main(String[] args) throws MQClientException {
		DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("gzy_consumer");
		// 设置consumer第一次启动是从队列头部开始还是尾部开始消费，若非第一次启动，那么按照上次消费的位置继续消费
		consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_FIRST_OFFSET);
		consumer.subscribe("TopicQuickStart", "*");
		// 批量消费，一次消费多少条消息，默认为1条,最大情况能拿多少条不代表每次能拿这么多条
		// consumer.setConsumeMessageBatchMaxSize(3);
		consumer.registerMessageListener(new MessageListenerConcurrently() {
		//如果没有Override，编译为jdk1.5需要改回来
			@Override
			public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) {
				// System.out.println(Thread.currentThread().getName() + "Receive: " + msgs);
				// 获取一次性消费多少条消息
				// System.out.println("消息条数 ： " + msgs.size());
				MessageExt msg1 = null;
				try {
					for (MessageExt msg : msgs) {
						msg1 = msg;
						String topic = msg.getTopic();
						String msgbody = new String(msg.getBody(), "utf-8");
						String tag = msg.getTags();
						System.out.println("收到消息： " + "topic:" + topic + " tags:" + tag + " msg:" + msgbody);
					}
				} catch (Exception e) {
					e.printStackTrace();
					// 若已经重试了5次则不再重试
					if (msg1.getReconsumeTimes() == 5) {
						// 此处记录日志操作。。。
						return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
					}
					return ConsumeConcurrentlyStatus.RECONSUME_LATER;
				}

				return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
			}
		});
		consumer.setNamesrvAddr("localhost:9876");
		consumer.start();
		System.out.println("Consumer started...");
	}

}

先运行发布端，控制台

SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E83EF0000, offsetMsgId=C0A8586500002A9F0000000000001DC4, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=0], queueOffset=10]
SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E83F90001, offsetMsgId=C0A8586500002A9F0000000000001E82, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=1], queueOffset=10]
SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E83FB0002, offsetMsgId=C0A8586500002A9F0000000000001F40, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=2], queueOffset=10]
SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E83FC0003, offsetMsgId=C0A8586500002A9F0000000000001FFE, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=3], queueOffset=10]
SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E83FE0004, offsetMsgId=C0A8586500002A9F00000000000020BC, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=0], queueOffset=11]
SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E84000005, offsetMsgId=C0A8586500002A9F000000000000217A, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=1], queueOffset=11]
SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E84020006, offsetMsgId=C0A8586500002A9F0000000000002238, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=2], queueOffset=11]
SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E84030007, offsetMsgId=C0A8586500002A9F00000000000022F6, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=3], queueOffset=11]
SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E84050008, offsetMsgId=C0A8586500002A9F00000000000023B4, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=0], queueOffset=12]
SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E84070009, offsetMsgId=C0A8586500002A9F0000000000002472, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=1], queueOffset=12]
SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E8408000A, offsetMsgId=C0A8586500002A9F0000000000002530, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=2], queueOffset=12]
SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E840A000B, offsetMsgId=C0A8586500002A9F00000000000025EF, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=3], queueOffset=12]
SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E840C000C, offsetMsgId=C0A8586500002A9F00000000000026AE, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=0], queueOffset=13]
SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E8410000D, offsetMsgId=C0A8586500002A9F000000000000276D, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=1], queueOffset=13]
SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E8413000E, offsetMsgId=C0A8586500002A9F000000000000282C, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=2], queueOffset=13]
SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E8415000F, offsetMsgId=C0A8586500002A9F00000000000028EB, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=3], queueOffset=13]
SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E84170010, offsetMsgId=C0A8586500002A9F00000000000029AA, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=0], queueOffset=14]
SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E84180011, offsetMsgId=C0A8586500002A9F0000000000002A69, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=1], queueOffset=14]
SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E841A0012, offsetMsgId=C0A8586500002A9F0000000000002B28, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=2], queueOffset=14]
SendResult [sendStatus=SEND_OK, msgId=C0A85865391A15DB9742545E841C0013, offsetMsgId=C0A8586500002A9F0000000000002BE7, messageQueue=MessageQueue [topic=TopicQuickStart, brokerName=macdeMacBook-Pro.local, queueId=3], queueOffset=14]

再运行消费者端

Consumer started...
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波7
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波6
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波2
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波15
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波11
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波5
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波14
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波10
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波0
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波3
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波13
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波9
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波4
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波1
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波19
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波8
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波12
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波18
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波17
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波16
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波0
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波1
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波2
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波3
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波4
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波5
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波6
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波7
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波8
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波9
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波10
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波11
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波12
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波13
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波14
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波15
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波16
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波17
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波18
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波19
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波0
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波1
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波2
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波3
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波4
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波5
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波6
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波7
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波8
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波9
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波10
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波11
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波12
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波13
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波14
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波15
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波16
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波17
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波18
收到消息： topic:TopicQuickStart tags:gzy_tag msg:发射动感光波19

发送和接收都成功～

3. 可视化工具使用

rocketMQ的开源web可是化工具是 RocketMQ console。

  1. 先安装控制台web应用
$ git clone -b release-rocketmq-console-1.0.0 https://github.com/apache/rocketmq-externals.git
#进入配置文件修改配置
$ cd rocketmq-externals/rocketmq-console/
$ vi src/main/resources/application.properties

或者直接去这个文件夹下编辑该文件，由该配置文件可知，控制台是个spring应用

#管理后台访问上下文路径，默认为空，如果填写，一定要前面加“/”，后面不要加，否则启动报错
server.contextPath=/rocketMQ
server.port=8080
#spring.application.index=true
spring.application.name=rocketmq-console
spring.http.encoding.charset=UTF-8
spring.http.encoding.enabled=true
spring.http.encoding.force=true
logging.config=classpath:logback.xml
#if this value is empty,use env value rocketmq.config.namesrvAddr  NAMESRV_ADDR | now, you can set it in ops page.default localhost:9876
rocketmq.config.namesrvAddr=
#if you use rocketmq version < 3.5.8, rocketmq.config.isVIPChannel should be false.default true
rocketmq.config.isVIPChannel=
#rocketmq-console's data path:dashboard/monitor
rocketmq.config.dataPath=/tmp/rocketmq-console/data
#set it false if you don't want use dashboard.default true
rocketmq.config.enableDashBoardCollect=true
  1. 跑起来
将项目打成jar包，并运行jar文件。也可以用ide导入工程启动
$ mvn clean package -Dmaven.test.skip=true
$ java -jar target/rocketmq-console-ng-1.0.0.jar
#如果配置文件没有填写Name Server
$ java -jar target/rocketmq-console-ng-1.0.0.jar --rocketmq.config.namesrvAddr='10.0.74.198:9876;10.0.74.199:9876'

我这里是直接导到 eclipse里运行的，是个springboot项目，主程序是App.java

可以查看集群、生产者和消费者等信息。

转载于:https://my.oschina.net/xlpapapa/blog/3010374

weixin_34268753
关注
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值