1.rabbitmq简介
MQ全称为Message Queue, 消息队列(MQ)是一种应用程序对应用程序的通信方法。应用程序通过读写出入队列的消息(针对应用程序的数据)来通信,而无需专用连接来链接它们。消息传递指的是程序之间通过在消息中发送数据进行通信,而不是通过直接调用彼此来通信,直接调用通常是用于诸如远程过程调用的技术。排队指的是应用程序通过 队列来通信。队列的使用除去了接收和发送应用程序同时执行的要求。其中较为成熟的MQ产品有IBM WEBSPHERE MQ等等。
2.应用场景
在项目中,将一些无需即时返回且耗时的操作提取出来,进行了异步处理,而这种异步处理的方式大大的节省了服务器的请求响应时间,从而提高了系统的吞吐量。
3.rabbitmq是一个由erlang(面向并发的编程语言)开发的AMQP(Advanced Message Queue消息队列)的开源实现,主要包含以下组件:
1)server(broker):接受客户端连接,实现AMQP消息队列和路由功能的进程。
2)Virtual Host:其实是一个虚拟概念,类似于权限控制组,一个Virtual Host里面可以有若干个Exchange和Queue,但是权限控制的最小粒度是Virtual Host。
3)Exchange:接收生产者发送的消息,并根据Binding规则将消息路由给服务器中的队列。ExchangeType决定了Exchange路由消息的行为,例如:在Rabbitmq中,ExchangeType有direct、fanout和topic三种,不同类型的Exchange路由的行为是不一样的。
4)Message Queue:消息队列,用于存储还未被消费者消费的消息。
5)Message:有Header和Body组成,Header由生产者添加各种属性的集合,包括Message是否被持久化,由那个Message Queue接受、优先级是多少等。而Body是真正需要传输的APP数据。
6)Binding:Binding联系了Exchange与Message Queue。Exchange在与多个Message Queue发生Binding后会生成一张路由表,路由表中存储着Message Queue所需消息的限制条件即Binding Key。当Exchange收到Message时会解析其Header得到Routing Key,Exchange根据Routing Key与Exchange Type将Message路由到Message Queue。Binding Key由Consumer在Binding Exchange与Message Queue时指定,而Routing Key由Producer发送Message时指定,两者的匹配方式由Exchange Type决定。
7)Connection:连接,对于RabbitMQ而言,其实就是一个位于客户端和Broker之间的TCP连接。
8)Channel:信道,仅仅创建了客户端到Broker之间的连接后,客户端还是不能发送消息的。需要为每一个Connection创建Channel,AMQP协议规定只有通过Channel才能执行AMQP的命令。一个Connection可以包含多个Channel。之所以需要Channel,是因为TCP连接的建立和释放都是十分昂贵的,如果一个客户端每一个线程都需要与Broker交互,如果每一个线程都建立一个TCP连接,暂且不考虑TCP连接是否浪费,就算操作系统也无法承受每秒建立如此多的TCP连接。RabbitMQ建议客户端线程之间不要共用Channel,至少要保证共用Channel的线程发送消息必须是串行的,但是建议尽量共用Connection。
9).Command:AMQP的命令,客户端通过Command完成与AMQP服务器的交互来实现自身的逻辑。例如在RabbitMQ中,客户端可以通过publish命令发送消息,txSelect开启一个事务,txCommit提交一个事务。
4.安装
参考:http://www.rabbitmq.com/download.html
1)yum安装(可以将所有依赖的包一起安装):
rabbitmq-server 目前安装包被包含在 Fedora rpm仓库中 Fedora是epel库
先安装epel私服:yum -y install epel-release.noarch
搜索rabbitmq的server:yum search rabbitmq-server
搜索到后安装:yum -y install rabbitmq-server.noarch
安装后可查看是否安装成功:rpm -ql rabbitmq-server
Rabbitmq默认安装后会添加账号rabbitmq默认以该账号运行:more /etc/passwd | grep rabbitmq
进入服务的路径:cd /etc/init.d(相当于Windows中的services.msc)
2) 启动rabbitmq:service rabbitmq-server start
启动后查看默认端口5672:netstat -aon | grep 5672
查看运行状态:service rabbitmq-server status
默认的日志目录:/var/log/rabbitmq
rabbitmq默认提供了一个web管理工具(rabbitmq_management)参考官方http://www.rabbitmq.com/management.html 默认已经安装 是一个插件,查看所有插件:rabbitmq-plugins list
启用该插件:./rabbitmq-plugins enable rabbitmq_management
重启rabbitmq-server:service rabbitmq-server restart
关闭rabbitmq:rabbitmqctl stop
查看已安插件:rpm -ql rabbitmq-server | grep rabbitmq-plugins-->cd /usr/lib/rabbitmq/bin-->./rabbitmq-plugins list
3)Web界面监控rabbitmq服务:浏览器访问(开启了端口 15672 当前机器ip是6.128) (http://192.168.6.128:15672/)
输入用户名 guest和guest
5. rabbitmq支持6种消息接受和转发机制
Jar包依赖:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>cn.et</groupId> <artifactId>RABBITMQ_PUB</artifactId> <version>0.0.1-SNAPSHOT</version>
<!-- 添加rabbitmq依赖 --> <dependencies> <!-- https://mvnrepository.com/artifact/com.rabbitmq/amqp-client --> <dependency> <groupId>com.rabbitmq</groupId> <artifactId>amqp-client</artifactId> <version>4.2.0</version> </dependency>
</dependencies> </project> |
1)简单模式:(http://www.rabbitmq.com/tutorials/tutorial-one-java.html)
单个发送者(生产者)将消息发送到队列(每个队列都有一个唯一的名字) 单个接收者(消费者)获取消息
测试代码结构:
测试代码:
SendEmail.java:
package cn.et;
import java.io.ByteArrayInputStream; import java.io.ByteArrayOutputStream; import java.io.IOException; import java.io.ObjectInputStream; import java.io.ObjectOutputStream; import java.util.Map; import java.util.concurrent.TimeoutException;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.mail.SimpleMailMessage; import org.springframework.mail.javamail.JavaMailSender; import org.springframework.stereotype.Controller; import org.springframework.web.bind.annotation.RequestMapping;
import com.rabbitmq.client.AMQP; import com.rabbitmq.client.Channel; import com.rabbitmq.client.Connection; import com.rabbitmq.client.ConnectionFactory; import com.rabbitmq.client.Consumer; import com.rabbitmq.client.DefaultConsumer; import com.rabbitmq.client.Envelope; @Controller public class SendEmail { /** * 序列化(对象转字节数组) * @return * @throws IOException */ public byte[] seq(Object obj) throws IOException{ ByteArrayOutputStream bos = new ByteArrayOutputStream(); ObjectOutputStream oos = new ObjectOutputStream(bos); oos.writeObject(obj); return bos.toByteArray(); } /** * 反序列化(字节数组转对象) * @param bt * @return * @throws IOException * @throws ClassNotFoundException */ public static Object dseq(byte[] bt) throws IOException, ClassNotFoundException{ ByteArrayInputStream bis = new ByteArrayInputStream(bt); ObjectInputStream ois = new ObjectInputStream(bis); return ois.readObject(); } private static final String QUEUE_NAME = "MAIL_QUEUE"; @Autowired private JavaMailSender jms ; @RequestMapping("/send") public void send() throws IOException, TimeoutException{ ConnectionFactory factory = new ConnectionFactory(); factory.setHost("192.168.6.128"); Connection connection = factory.newConnection(); Channel channel = connection.createChannel(); //消费者也需要定义队列 有可能消费者先于生产者启动 channel.queueDeclare(QUEUE_NAME, false, false, false, null); //定义回调抓取消息 Consumer consumer = new DefaultConsumer(channel){ @Override public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException { try { Map map = (Map)dseq(body); System.out.println(jms+"------------------------------"+map); SimpleMailMessage smm = new SimpleMailMessage(); smm.setFrom("sunyingida@126.com"); smm.setTo(map.get("sendTo").toString()); smm.setSubject(map.get("subject").toString()); smm.setText(map.get("content").toString()); System.out.println(map+"----------"); System.out.println(jms); jms.send(smm); System.out.println(map.get("content")); } catch (ClassNotFoundException e) { e.printStackTrace(); } catch (Exception e) { // TODO Auto-generated catch block e.printStackTrace(); }
} }; //发送邮件 channel.basicConsume(QUEUE_NAME, true, consumer); } }
|
Mail_Cons.java:
package cn.et;
import java.io.IOException; import java.util.concurrent.TimeoutException;
import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication public class Mail_Cons { public static void main(String[] args) throws IOException, TimeoutException { SpringApplication.run(Mail_Cons.class, args); }
}
|
Pub_Email.java:
package cn.et;
import java.io.ByteArrayInputStream; import java.io.ByteArrayOutputStream; import java.io.IOException; import java.io.ObjectInputStream; import java.io.ObjectOutputStream; import java.util.HashMap; import java.util.Map; import java.util.concurrent.TimeoutException;
import com.rabbitmq.client.Channel; import com.rabbitmq.client.Connection; import com.rabbitmq.client.ConnectionFactory;
public class Pub_Email { /** * 序列化(将对象转换成字节数组) * @return * @throws IOException */ public static byte[] seq(Object obj) throws IOException{ ByteArrayOutputStream bos = new ByteArrayOutputStream(); ObjectOutputStream oos = new ObjectOutputStream(bos); oos.writeObject(obj); return bos.toByteArray(); } /** * 反序列化(将字节数组转成对象) * @param bt * @return * @throws IOException * @throws ClassNotFoundException */ public static Object dseq(byte[] bt) throws IOException, ClassNotFoundException{ ByteArrayInputStream bis = new ByteArrayInputStream(bt); ObjectInputStream ois = new ObjectInputStream(bis); return ois.readObject(); } private final static String QUEUE_NAME = "MAIL_QUEUE"; public static void main(String[] args) throws IOException, TimeoutException { //(模拟发送邮件)创建发送邮件的任务 Map map = new HashMap(); map.put("sendTo", "sunyingida@163.com"); map.put("subject", "测试邮件"); map.put("content", "啦啦啦"); //连接到远程rabbitmq服务器 ConnectionFactory factory = new ConnectionFactory(); factory.setHost("192.168.6.128"); factory.setPort(5672); Connection connection = factory.newConnection(); Channel channel = connection.createChannel(); //创建一个队列存储邮件信息 channel.queueDeclare(QUEUE_NAME, false, false, false, null); //发送消息 需要序列化 :seq(map) channel.basicPublish("", QUEUE_NAME, null, seq(map)); } } |
application.properties:
spring.mail.host=smtp.126.com spring.mail.username=sunyingida@126.com spring.mail.password=aaa123456 spring.mail.port=25 spring.mail.protocol=smtp eureka.client.serviceUrl.defaultZone=http://localhost:8761/eureka/ spring.application.name=SENDMAIL server.port=8080 |
2)工作队列模式:(http://www.rabbitmq.com/tutorials/tutorial-two-java.html)
Pub_Work.java:
package cn.et;
import java.io.IOException; import java.util.concurrent.TimeoutException;
import com.rabbitmq.client.Channel; import com.rabbitmq.client.Connection; import com.rabbitmq.client.ConnectionFactory;
public class Pub_Work { //给队列取个名 private static final String QUEUE_NAME = "WORK_QUEUE";
public static void main(String[] args) throws IOException, TimeoutException { //创建连接 ConnectionFactory factory = new ConnectionFactory(); factory.setHost("192.168.6.128"); factory.setPort(5672); Connection connection = factory.newConnection(); //创建管道 Channel channel = connection.createChannel(); //创建队列 channel.queueDeclare(QUEUE_NAME, false, false, false, null); //生产10个消息 for(int i=0;i<10;i++){ channel.basicPublish("", QUEUE_NAME, null,("这是:"+i).getBytes("UTF-8")); } channel.close(); connection.close(); } } |
Pub_Work.java:
package cn.et;
import java.io.IOException; import java.util.concurrent.TimeoutException;
import com.rabbitmq.client.AMQP; import com.rabbitmq.client.Channel; import com.rabbitmq.client.Connection; import com.rabbitmq.client.ConnectionFactory; import com.rabbitmq.client.Consumer; import com.rabbitmq.client.DefaultConsumer; import com.rabbitmq.client.Envelope;
public class Work_Cons { private static final String QUEUE_NAME = "WORK_QUEUE";
public static void main(String[] args) throws IOException, TimeoutException { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("192.168.6.128"); Connection connection = factory.newConnection(); Channel channel = connection.createChannel(); channel.queueDeclare(QUEUE_NAME,false , false, false, null); Consumer consumer = new DefaultConsumer(channel){ @Override public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException { System.out.println(new String(body, "UTF-8"));
} }; channel.basicConsume(QUEUE_NAME, true,consumer); }
}
|
3)发布订阅模式:(http://www.rabbitmq.com/tutorials/tutorial-three-java.html)
Pub_Log.java:
package cn.et;
import java.io.IOException; import java.util.concurrent.TimeoutException; import com.rabbitmq.client.Channel; import com.rabbitmq.client.Connection; import com.rabbitmq.client.ConnectionFactory; import com.rabbitmq.client.MessageProperties;
public class Pub_Log { private static final String EXCHANGE_NAME = "amq_log";
/** * 发布订阅模式 * @param args * @throws TimeoutException * @throws IOException */ public static void main(String[] args) throws IOException, TimeoutException { //创建连接 ConnectionFactory factory = new ConnectionFactory(); factory.setHost("192.168.6.128"); factory.setPort(5672); Connection connection = factory.newConnection(); Channel channel = connection.createChannel(); //创建一个交换器 (交换器名字,交换器类型,是否将交换器信息永久保存在服务器磁盘上) //发布订阅模式"fanout" channel.exchangeDeclare(EXCHANGE_NAME, "fanout",true); String message = null; //同时发送5条消息 for(int i=0;i<5;i++){ message = "第"+i+"条消息"; channel.basicPublish(EXCHANGE_NAME, "",MessageProperties.PERSISTENT_TEXT_PLAIN,message.getBytes("UTF-8")); } channel.close(); connection.close(); } } |
Log_Cons.java:
package cn.et;
import java.io.IOException; import java.util.concurrent.TimeoutException;
import com.rabbitmq.client.AMQP; import com.rabbitmq.client.Channel; import com.rabbitmq.client.Connection; import com.rabbitmq.client.ConnectionFactory; import com.rabbitmq.client.Consumer; import com.rabbitmq.client.DefaultConsumer; import com.rabbitmq.client.Envelope;
public class Log_Cons { private static final String EXCHANGE_NAME = "amq_log"; public static void main(String[] args) throws IOException, TimeoutException { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("192.168.6.128"); Connection connection = factory.newConnection(); Channel channel = connection.createChannel(); channel.exchangeDeclare(EXCHANGE_NAME,"fanout", true); //定义一个随机的队列 用于交换器获取消息 String queueName = channel.queueDeclare().getQueue(); channel.queueBind(queueName, EXCHANGE_NAME, ""); Consumer consumer = new DefaultConsumer(channel){ @Override public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException { System.out.println(new String(body, "UTF-8")); } }; channel.basicConsume(queueName, false,consumer); } } |
4)路由模式:(http://www.rabbitmq.com/tutorials/tutorial-three-java.html)
Pub_Log_Route.java:
package cn.et;
import java.io.IOException; import java.util.concurrent.TimeoutException;
import com.rabbitmq.client.Channel; import com.rabbitmq.client.Connection; import com.rabbitmq.client.ConnectionFactory; import com.rabbitmq.client.MessageProperties;
public class Pub_Log_Route {
private static final String EXCHANGE_NAME = "amq_log_route";
/** * 路由模式 direct * @param args * @throws TimeoutException * @throws IOException */ public static void main(String[] args) throws IOException, TimeoutException { //创建连接 ConnectionFactory factory = new ConnectionFactory(); factory.setHost("192.168.6.128"); factory.setPort(5672); Connection connection = factory.newConnection(); Channel channel = connection.createChannel(); //创建一个交换器 (交换器名字,交换器类型,是否将交换器信息永久保存在服务器磁盘上) //路由模式 类型:"direct" channel.exchangeDeclare(EXCHANGE_NAME, "direct",true); //:routekey:error channel.basicPublish(EXCHANGE_NAME, "error",MessageProperties.PERSISTENT_TEXT_PLAIN,("系统正在维护...").getBytes("UTF-8")); channel.basicPublish(EXCHANGE_NAME, "info",MessageProperties.PERSISTENT_TEXT_PLAIN,("张三访问了页面2018-1-11").getBytes("UTF-8")); channel.basicPublish(EXCHANGE_NAME, "info",MessageProperties.PERSISTENT_TEXT_PLAIN,("李四访问了页面2018-1-11").getBytes("UTF-8")); channel.close(); connection.close(); }
} |
Log_Cons_Route.java:
package cn.et;
import java.io.IOException; import java.util.concurrent.TimeoutException;
import com.rabbitmq.client.AMQP; import com.rabbitmq.client.Channel; import com.rabbitmq.client.Connection; import com.rabbitmq.client.ConnectionFactory; import com.rabbitmq.client.Consumer; import com.rabbitmq.client.DefaultConsumer; import com.rabbitmq.client.Envelope;
public class Log_Cons_Route { private static final String EXCHANGE_NAME = "amq_log_route";
public static void main(String[] args) throws IOException, TimeoutException { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("192.168.6.128"); Connection connection = factory.newConnection(); Channel channel = connection.createChannel(); channel.exchangeDeclare(EXCHANGE_NAME,"direct", true); //定义一个随机的队列 用于交换器获取消息 String queueName = channel.queueDeclare().getQueue(); //指定接收哪些消息 channel.queueBind(queueName, EXCHANGE_NAME, "info"); Consumer consumer = new DefaultConsumer(channel){ @Override public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException { System.out.println(new String(body, "UTF-8"));
} }; channel.basicConsume(queueName, false,consumer); } } |
5)Topics路由模式:(http://www.rabbitmq.com/tutorials/tutorial-three-java.html)
Pub_Log_TopicRoute.java:
package cn.et;
import java.io.IOException; import java.util.concurrent.TimeoutException;
import com.rabbitmq.client.Channel; import com.rabbitmq.client.Connection; import com.rabbitmq.client.ConnectionFactory; import com.rabbitmq.client.MessageProperties;
public class Pub_Log_TopicRoute {
private static final String EXCHANGE_NAME = "amq_log_topic";
/** * 路由模式 topic * @param args * @throws TimeoutException * @throws IOException */ public static void main(String[] args) throws IOException, TimeoutException { //创建连接 ConnectionFactory factory = new ConnectionFactory(); factory.setHost("192.168.6.128"); factory.setPort(5672); Connection connection = factory.newConnection(); Channel channel = connection.createChannel(); //创建一个交换器 (交换器名字,交换器类型,是否将交换器信息永久保存在服务器磁盘上) //路由模式 类型:"direct" channel.exchangeDeclare(EXCHANGE_NAME, "topic",true); //:routekey:error /*channel.basicPublish(EXCHANGE_NAME, "a.error",MessageProperties.PERSISTENT_TEXT_PLAIN,("系统正在维护...").getBytes("UTF-8")); channel.basicPublish(EXCHANGE_NAME, "a.info",MessageProperties.PERSISTENT_TEXT_PLAIN,("张三访问了页面2018-1-11").getBytes("UTF-8")); channel.basicPublish(EXCHANGE_NAME, "a.info",MessageProperties.PERSISTENT_TEXT_PLAIN,("李四访问了页面2018-1-11").getBytes("UTF-8")); channel.basicPublish(EXCHANGE_NAME, "b.error",MessageProperties.PERSISTENT_TEXT_PLAIN,("系统正在维护...").getBytes("UTF-8")); channel.basicPublish(EXCHANGE_NAME, "b.info",MessageProperties.PERSISTENT_TEXT_PLAIN,("张三访问了页面2018-1-11").getBytes("UTF-8")); channel.basicPublish(EXCHANGE_NAME, "b.info",MessageProperties.PERSISTENT_TEXT_PLAIN,("李四访问了页面2018-1-11").getBytes("UTF-8")); */ channel.basicPublish(EXCHANGE_NAME, "a.a1.error",MessageProperties.PERSISTENT_TEXT_PLAIN,("系统正在维护...").getBytes("UTF-8")); channel.basicPublish(EXCHANGE_NAME, "a.a2.info",MessageProperties.PERSISTENT_TEXT_PLAIN,("张三访问了页面2018-1-11").getBytes("UTF-8")); channel.basicPublish(EXCHANGE_NAME, "a.a3.info",MessageProperties.PERSISTENT_TEXT_PLAIN,("李四访问了页面2018-1-11").getBytes("UTF-8")); channel.basicPublish(EXCHANGE_NAME, "b.b1.error",MessageProperties.PERSISTENT_TEXT_PLAIN,("系统正在维护...").getBytes("UTF-8")); channel.basicPublish(EXCHANGE_NAME, "b.b2.info",MessageProperties.PERSISTENT_TEXT_PLAIN,("张三访问了页面2018-1-11").getBytes("UTF-8")); channel.basicPublish(EXCHANGE_NAME, "b.b3.info",MessageProperties.PERSISTENT_TEXT_PLAIN,("李四访问了页面2018-1-11").getBytes("UTF-8"));
channel.close(); connection.close(); }
} |
Log_Cons_TopicRoute:
package cn.et;
import java.io.IOException; import java.util.concurrent.TimeoutException;
import com.rabbitmq.client.AMQP; import com.rabbitmq.client.Channel; import com.rabbitmq.client.Connection; import com.rabbitmq.client.ConnectionFactory; import com.rabbitmq.client.Consumer; import com.rabbitmq.client.DefaultConsumer; import com.rabbitmq.client.Envelope;
public class Log_Cons_TopicRoute { private static final String EXCHANGE_NAME = "amq_log_topic";
public static void main(String[] args) throws IOException, TimeoutException { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("192.168.6.128"); Connection connection = factory.newConnection(); Channel channel = connection.createChannel(); channel.exchangeDeclare(EXCHANGE_NAME,"topic", true); //定义一个随机的队列 用于交换器获取消息 String queueName = channel.queueDeclare().getQueue(); //指定接收哪些消息 *匹配一层 #匹配多层 *代表两个.中的一段,#代表多段 //channel.queueBind(queueName, EXCHANGE_NAME, "b.*"); channel.queueBind(queueName, EXCHANGE_NAME, "a.#"); Consumer consumer = new DefaultConsumer(channel){ @Override public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException { System.out.println(new String(body, "UTF-8")); } }; channel.basicConsume(queueName, false,consumer); } } |
6)RPC模式:
RPC工作方式:
1)当客户端启动时,会创建一个匿名的回调队列
2)在RPC请求中,定义了两个属性:replyTo,表示回调队列的名称; correlationId,表示请求任务的唯一编号,用来区分3)不同请求的返回结果。
4)将请求发送到rpc_queue队列中
5)RPC服务器等待rpc_queue队列的请求,如果有消息,就处理,它将计算结果发送到请求中的回调队列里。
6)客户端监听回调队列中的消息,如果有返回消息,它根据回调消息中的correlationid进行匹配计算结果。