我们经常在Spring项目中或者Spring Boot项目中使用RabbitMQ,一般使用的时候,己经由前人将配置配置好了,我们只需要写一个注解或者调用一个消息发送或者接收消息的监听器即可,但是底层是如何实现的呢?大部分开发者来说,基本上是一头雾水,基本上不知道如何实现。在读本文博客的时候,如果对RabbitMQ的基本概念不太懂的同学,可以先去学习我的https://blog.csdn.net/quyixiao/article/details/116025823 这一篇博客,主要讲RabbitMQ信道,交换器,队列等基本概念,以及使用Java 客户端生产者消费者各个场景的使用。不然,直接来看Spring整合RabbitMQ的源码实现,可能有很多知识点觉得莫名其妙,因此,当你来看这篇博客时,我觉得你是对RabbitMQ有一定了解的小伙伴,话不多说,直接来看一个简单的示例,我们将从这个简单的示例,来剖析整个Spring整合RabbitMQ源码实现。
- rabbitMQ生产者配置文件。
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:rabbit="http://www.springframework.org/schema/rabbit" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd http://www.springframework.org/schema/rabbit http://www.springframework.org/schema/rabbit/spring-rabbit-1.0.xsd"> <rabbit:connection-factory id="connectionFactory" host="172.16.157.242" username="guest" password="guest" port="5672" virtual-host="/"/> <rabbit:admin connection-factory="connectionFactory"/> <!--申明一个消息队列Queue durable:是否持久化 exclusive: 仅创建者可以使用的私有队列,断开后自动删除 auto_delete: 当所有消费客户端连接断开后,是否自动删除队列--> <rabbit:queue id="test_queue_key" name="test_queue_key" durable="true" auto-delete="false" exclusive="false"/> <!-- 声明Exchange rabbit:direct-exchange:定义exchange模式为direct --> <rabbit:direct-exchange name="test-mq-exchange" durable="true" auto-delete="false" id="test-mq-exchange"> <rabbit:bindings> <rabbit:binding queue="test_queue_key" key="test_queue_key"/> </rabbit:bindings> </rabbit:direct-exchange> <!-- spring template声明--> <rabbit:template id="rabbitTemplate" exchange="test-mq-exchange" connection-factory="connectionFactory" message-converter="jsonMessageConverter"/> <!-- 消息对象json转换类 --> <bean id="jsonMessageConverter" class="org.springframework.amqp.support.converter.Jackson2JsonMessageConverter"/> </beans>
- rabbitMQ生产者测试
public class MQSender { public static final String queue_key = "test_queue_key"; public static void main(String[] args) { ApplicationContext context = new ClassPathXmlApplicationContext("classpath:spring_1_100/config_71_80/spring78_rabbitmq/spring78_mq_sender.xml"); Map<String, Object> msg = new HashMap(); msg.put("data", "hello,rabbmitmq!"); msg.put("time", System.currentTimeMillis()); System.out.println("++++++++++++++++++++" + JSON.toJSONString(msg)); RabbitTemplate rabbitTemplate = (RabbitTemplate) context.getBean("rabbitTemplate"); rabbitTemplate.convertAndSend(queue_key, msg); } }
- rabbitMQ 消费者配置文件
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:rabbit="http://www.springframework.org/schema/rabbit" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd http://www.springframework.org/schema/rabbit http://www.springframework.org/schema/rabbit/spring-rabbit-1.0.xsd"> <description>rabbitmq 连接服务配置</description> <!-- 连接配置 --> <rabbit:connection-factory id="connectionFactory" host="172.16.157.242" username="guest" password="guest" port="5672" virtual-host="/"/> <!--申明一个消息队列Queue durable:是否持久化 exclusive: 仅创建者可以使用的私有队列,断开后自动删除 auto_delete: 当所有消费客户端连接断开后,是否自动删除队列--> <rabbit:queue id="test_queue_key" name="test_queue_key" durable="true" auto-delete="false" exclusive="false"/> <rabbit:listener-container connection-factory="connectionFactory" acknowledge="auto"> <rabbit:listener queues="test_queue_key" ref="queueListenter"/> </rabbit:listener-container> <bean id="queueListenter" class="com.spring_1_100.test_71_80.test78_spring_rabbitmq.spring.QueueListenter"></bean> </beans>
- rabbitMQ 消费者监听器
public class QueueListenter implements MessageListener { @Override public void onMessage(Message message) { System.out.println("---------------------" + message); } }
- rabbitMQ 消费者测试
public class MQReciver { public static void main(String[] args) { new ClassPathXmlApplicationContext("classpath:spring_1_100/config_71_80/spring78_rabbitmq/spring78_mq_recive.xml"); } }
测试:
发送端日志:
++++++++++++++++++++{“data”:“hello,rabbmitmq!”,“time”:1619167470817}
接收端日志:
---------------------(Body:’{“data”:“hello,rabbmitmq!”,“time”:1619167470817}'MessageProperties [headers={ContentTypeId=java.lang.Object, KeyTypeId=java.lang.Object, TypeId=java.util.HashMap}, timestamp=null, messageId=null, userId=null, appId=null, clusterId=null, type=null, correlationId=null, replyTo=null, contentType=application/json, contentEncoding=UTF-8, contentLength=0, deliveryMode=PERSISTENT, expiration=null, priority=0, redelivered=false, receivedExchange=test-mq-exchange, receivedRoutingKey=test_queue_key, deliveryTag=1, messageCount=0])
从测试结果来看,我们看到了生产者发送一条消息,被消费者消费。那是怎样实现的呢?带着疑问,我们来看看源码,首先,来看Spring是如何解析配置文件的。我们先来找到标签解析器
可能有人会觉得,你怎么知道xsd所在项目的META-INF下的spring.handles文件中就有相应标签的解析器呢?这个是我读取Spring源码总结得到的经验,一般就按这个方法找,就可找到标签解析器。我们来看看spring.handlers文件中配置了什么呢?
http://www.springframework.org/schema/rabbit=org.springframework.amqp.rabbit.config.RabbitNamespaceHandler
在配置文件中,我们看到了上面这一行配置,那我们进入RabbitNamespaceHandler中,看内部写了什么。
public class RabbitNamespaceHandler extends NamespaceHandlerSupport { public void init() { registerBeanDefinitionParser("queue", new QueueParser()); registerBeanDefinitionParser("direct-exchange", new DirectExchangeParser()); registerBeanDefinitionParser("topic-exchange", new TopicExchangeParser()); registerBeanDefinitionParser("fanout-exchange", new FanoutExchangeParser()); registerBeanDefinitionParser("headers-exchange", new HeadersExchangeParser()); registerBeanDefinitionParser("listener-container", new ListenerContainerParser()); registerBeanDefinitionParser("admin", new AdminParser()); registerBeanDefinitionParser("connection-factory", new ConnectionFactoryParser()); registerBeanDefinitionParser("template", new TemplateParser()); registerBeanDefinitionParser("queue-arguments", new QueueArgumentsParser()); registerBeanDefinitionParser("annotation-driven", new AnnotationDrivenParser()); } }
从字面意思上可以看出QueueParser即是队列解析器,DirectExchangeParser,TopicExchangeParser,FanoutExchangeParser,HeadersExchangeParser分别对四种类型交换器direct,fanout,topic和headers进行解析了,而QueueArgumentsParser肯定是对下面这些参数进行解析了
- Message TTL(x-message-ttl):设置队列中的所有消息的生存周期(统一为整个队列的所有消息设置生命周期), 也可以在发布消息的时候单独为某个消息指定剩余生存时间,单位毫秒, 类似于redis中的ttl,生存时间到了,消息会被从队里中删除,注意是消息被删除,而不是队列被删除, 特性Features=TTL, 单独为某条消息设置过期时间AMQP.BasicProperties.Builder properties = new AMQP.BasicProperties().builder().expiration(“6000”);
channel.basicPublish(EXCHANGE_NAME, “”, properties.build(), message.getBytes(“UTF-8”)); - Auto Expire(x-expires): 当队列在指定的时间没有被访问(consume, basicGet, queueDeclare…)就会被删除,Features=Exp
- Max Length(x-max-length): 限定队列的消息的最大值长度,超过指定长度将会把最早的几条删除掉, 类似于mongodb中的固定集合,例如保存最新的100条消息, Feature=Lim
Max Length Bytes(x-max-length-bytes): 限定队列最大占用的空间大小, 一般受限于内存、磁盘的大小, Features=Lim B - Dead letter exchange(x-dead-letter-exchange): 当队列消息长度大于最大长度、或者过期的等,将从队列中删除的消息推送到指定的交换机中去而不是丢弃掉,Features=DLX
Dead letter routing key(x-dead-letter-routing-key):将删除的消息推送到指定交换机的指定路由键的队列中去, Feature=DLK - Maximum priority(x-max-priority):优先级队列,声明队列时先定义最大优先级值(定义最大值一般不要太大),在发布消息的时候指定该消息的优先级, 优先级更高(数值更大的)的消息先被消费,
- Lazy mode(x-queue-mode=lazy): Lazy Queues: 先将消息保存到磁盘上,不放在内存中,当消费者开始消费的时候才加载到内存中
Master locator(x-queue-master-locator)
而AnnotationDrivenParser主要是对@EnableRabbit和@RabbitListener注解处理处理。当然TemplateParser让我们想到的就是JDBCTemplate,但RabbitMQ中的TemplateParser主要是为生产者使用,里面的配置主要是关于生产者的一些配置,如mandatory参数,当消息传递过程中不可达目的地时将消息返回给生产者的功能,事务消息,消息的confirm模式,重试template, 重试队列,重试监听器,先做一个简单的介绍,后面再来解释该如何使用这些配置,而ConnectionFactoryParser从字面意思就知道,是关于RabbitMQ的一些连接配置了。下面,我们来看我们测试用例中用到的解析器。
- ConnectionFactoryParser
class ConnectionFactoryParser extends AbstractSingleBeanDefinitionParser { @Override protected Class<?> getBeanClass(Element element) { return CachingConnectionFactory.class; } @Override protected boolean shouldGenerateId() { return false; } @Override protected boolean shouldGenerateIdAsFallback() { return true; } @Override protected void doParse(Element element, ParserContext parserContext, BeanDefinitionBuilder builder) { if (element.hasAttribute("addresses") && (element.hasAttribute(host) || element.hasAttribute(port))) { parserContext.getReaderContext().error("If the 'addresses' attribute is provided, a connection " + "factory can not have 'host' or 'port' attributes.", element); } NamespaceUtils.addConstructorArgParentRefIfAttributeDefined(builder, element, connection-factory); NamespaceUtils.setValueIfAttributeDefined(builder, element, "channel-cache-size"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "host"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "port"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "username"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "password"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "virtual-host"); NamespaceUtils.setReferenceIfAttributeDefined(builder, element, "executor"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "addresses"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "publisher-confirms"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "publisher-returns"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "requested-heartbeat", "requestedHeartBeat"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "connection-timeout"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "cache-mode"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "connection-cache-size"); } }
在上述源码中我们看到两个有意思的配置,publisher-confirms,publisher-returns 这两个配置有什么用呢?spring.rabbitmq.publisher-confirms=true ,设置此属性配置可以确保消息成功发送到交换器 ,而spring.rabbitmq.publisher-returns=true, 可以确保消息在未被队列接收时返回 ,publisher-return模式可以在消息没有被路由到指定的queue时将消息返回,而不是丢弃。但在使用上面的属性配置时通常会和mandatory属性配合一起使用。那口说无屏,先来看publisher-confirms的使用示例。
1. publisher-confirms使用示例
- 先创建生产者的Spring配置文件
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:rabbit="http://www.springframework.org/schema/rabbit" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd http://www.springframework.org/schema/rabbit http://www.springframework.org/schema/rabbit/spring-rabbit-1.4.xsd"> <!--spring.rabbitmq.publisher-confirms=true 设置此属性配置可以确保消息成功发送到交换器 , 1.publiser-confirm模式可以确保生产者到交换器exchange消息有没有发送成功--> <!--spring.rabbitmq.publisher-returns=true 可以确保消息在未被队列接收时返回 , 2.publisher-return模式可以在消息没有被路由到指定的queue时将消息返回,而不是丢弃--> <!-- spring.rabbitmq.template.mandatory=true , 指定消息在没有被队列接收时是否强行退回还是直接丢弃 , 在使用上面的属性配置时通常会和mandatory属性配合一起使用:--> <rabbit:connection-factory id="connectionFactory" host="172.16.157.242" username="guest" password="guest" port="5672" virtual-host="/" publisher-confirms="true" /> <rabbit:admin connection-factory="connectionFactory"/> <!--申明一个消息队列Queue durable:是否持久化 exclusive: 仅创建者可以使用的私有队列,断开后自动删除 auto_delete: 当所有消费客户端连接断开后,是否自动删除队列--> <!-- <rabbit:queue id="test_queue_key" name="test_queue_key" durable="true" auto-delete="false" exclusive="false" />--> <!-- 声明Exchange rabbit:direct-exchange:定义exchange模式为direct --> <!-- <rabbit:direct-exchange name="test-mq-exchange" durable="true" auto-delete="false" id="test-mq-exchange"> <rabbit:bindings> <!–<!–key 表示路由键–>–> <rabbit:binding queue="test_queue_key" key="test_queue_key"/> </rabbit:bindings> </rabbit:direct-exchange>--> <bean id="mqConfirmListener" class="com.spring_1_100.test_71_80.test78_spring_rabbitmq.spring1.MQConfirmListener"></bean> <!-- spring template声明--> <rabbit:template id="rabbitTemplate" exchange="test-mq-exchangexxxxxxxxxxxx" connection-factory="connectionFactory" mandatory="true" message-converter="jsonMessageConverter" confirm-callback="mqConfirmListener" > </rabbit:template> <!-- 消息对象json转换类 --> <bean id="jsonMessageConverter" class="org.springframework.amqp.support.converter.Jackson2JsonMessageConverter"/> </beans>
为了达到效果,注释掉所有的交换器及队列,并且在rabbittemplate中指定了一个不存在的交换器test-mq-exchangexxxxxxxxxxxx,设置mandatory为true,它们都有当消息传递过程中不可达目的地时将消息返回给生产者的功能。
- 创建生产者
public class MQSender { public static final String queue_key = "test_queue_keybbbbbbb"; public static void main(String[] args) { ApplicationContext context = new ClassPathXmlApplicationContext("classpath:spring_1_100/config_71_80/spring78_rabbitmq/spring1/spring78_mq_sender1.xml"); Map<String, Object> msg = new HashMap(); msg.put("data", "hello,rabbmitmq!"); msg.put("time", System.currentTimeMillis()); System.out.println("++++++++发送消息++++++++++++" + JSON.toJSONString(msg)); RabbitTemplate rabbitTemplate = (RabbitTemplate) context.getBean("rabbitTemplate"); rabbitTemplate.convertAndSend(queue_key, msg); } }
- 添加ConfirmListener
public class MQConfirmListener implements RabbitTemplate.ConfirmCallback { @Override public void confirm(CorrelationData correlationData, boolean ack, String cause) { if (ack) { System.out.println("--------消息没有被确认---------" + ack + " ,case " + cause); // 处理ack } else { // 处理nack, 此时cause包含nack的原因。 // 如当发送消息给一个不存在的Exchange。这种情况Broker会关闭Channel; // 当Broker关闭或发生网络故障时,需要重新发送消息。 // 暂时先日志记录,包括correlationData, cause等。 System.out.println("+++++++++++++++++" + ack + " ,case " + cause); } } }
【测试结果】
从上述结果中可以看出,当发布消息时,没有找到相应的交换器,在监听器中得到返回结果,及发送失败原因。
2. publisher-returns 使用示例
- 创建Spring配置文件
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:rabbit="http://www.springframework.org/schema/rabbit" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd http://www.springframework.org/schema/rabbit http://www.springframework.org/schema/rabbit/spring-rabbit-1.4.xsd"> <!--spring.rabbitmq.publisher-confirms=true 设置此属性配置可以确保消息成功发送到交换器 , 1.publiser-confirm模式可以确保生产者到交换器exchange消息有没有发送成功--> <!--spring.rabbitmq.publisher-returns=true 可以确保消息在未被队列接收时返回 , 2.publisher-return模式可以在消息没有被路由到指定的queue时将消息返回,而不是丢弃--> <!-- spring.rabbitmq.template.mandatory=true , 指定消息在没有被队列接收时是否强行退回还是直接丢弃 , 在使用上面的属性配置时通常会和mandatory属性配合一起使用:--> <rabbit:connection-factory id="connectionFactory" host="172.16.157.242" username="guest" password="guest" port="5672" virtual-host="/" publisher-returns="true" /> <rabbit:admin connection-factory="connectionFactory"/> <!--申明一个消息队列Queue durable:是否持久化 exclusive: 仅创建者可以使用的私有队列,断开后自动删除 auto_delete: 当所有消费客户端连接断开后,是否自动删除队列--> <!-- <rabbit:queue id="test_queue_key" name="test_queue_key" durable="true" auto-delete="false" exclusive="false" />--> <!-- 声明Exchange rabbit:direct-exchange:定义exchange模式为direct --> <rabbit:direct-exchange name="test-mq-exchangexxxxxxxxxxxx" durable="true" auto-delete="false" id="test-mq-exchangexxxxxxxxxxxx"> <!-- <rabbit:bindings> <!–key 表示路由键–> <rabbit:binding queue="test_queue_key" key="test_queue_key"/> </rabbit:bindings>--> </rabbit:direct-exchange> <bean id="mqConfirmListener" class="com.spring_1_100.test_71_80.test78_spring_rabbitmq.spring2_publisher_returns.MQConfirmListener"></bean> <!-- spring template声明--> <rabbit:template id="rabbitTemplate" exchange="test-mq-exchangexxxxxxxxxxxx" connection-factory="connectionFactory" mandatory="true" message-converter="jsonMessageConverter" return-callback="mqConfirmListener" > </rabbit:template> <!-- 消息对象json转换类 --> <bean id="jsonMessageConverter" class="org.springframework.amqp.support.converter.Jackson2JsonMessageConverter"/> </beans>
publisher-returns=“true”,表示如果消息发送到路由器中没有匹配对应的队列,则返回给指定的监听器。return-callback 指定未被匹配的消息处理器。mandatory需要指定为true。如果mandatory为false,消息将被丢弃掉。
- 依然创建生产者
public class MQSender { public static final String queue_key = "test_queue_keybbbbbbb"; public static void main(String[] args) { ApplicationContext context = new ClassPathXmlApplicationContext("classpath:spring_1_100/config_71_80/spring78_rabbitmq/spring2/spring78_mq_sender2.xml"); Map<String, Object> msg = new HashMap(); msg.put("data", "hello,rabbmitmq!"); msg.put("time", System.currentTimeMillis()); System.out.println("++++++++发送消息++++++++++++" + JSON.toJSONString(msg)); RabbitTemplate rabbitTemplate = (RabbitTemplate) context.getBean("rabbitTemplate"); rabbitTemplate.convertAndSend(queue_key, msg); } }
- 创建消息未被匹配到队列的接收器。
public class MQConfirmListener implements RabbitTemplate.ReturnCallback { @Override public void returnedMessage(Message message, int replyCode, String replyText, String exchange, String routingKey) { System.out.println("message = " + message); System.out.println("replyCode = " + replyCode + ",replyText="+replyText + ",exchange="+exchange + ",routingKey="+routingKey); } }
需要注意的是,publisher-confirms模式未被匹配的消息处理器是实现了RabbitTemplate.ConfirmCallback接口,而publisher-returns模式未被处理的消息处理器是实现了RabbitTemplate.ReturnCallback接口。
【测试结果】
上述关于如何实现,我们在后面的源码中再来分析了。申明了一个队列。那么队列是如何解析的呢?
public class QueueParser extends AbstractSingleBeanDefinitionParser { @Override protected boolean shouldGenerateIdAsFallback() { return true; } @Override protected Class<?> getBeanClass(Element element) { if (NamespaceUtils.isAttributeDefined(element, "name")) { return Queue.class; } else { return AnonymousQueue.class; } } @Override protected void doParse(Element element, ParserContext parserContext, BeanDefinitionBuilder builder) { //queue不允许不配置name或id,name和id至少需要配置一个或者两者都有 if (!NamespaceUtils.isAttributeDefined(element, "name") && !NamespaceUtils.isAttributeDefined(element, id)) { parserContext.getReaderContext().error("Queue must have either id or name (or both)", element); } NamespaceUtils.addConstructorArgValueIfAttributeDefined(builder, element, "name"); if (!NamespaceUtils.isAttributeDefined(element, "name")) { //如果队列没有name,表示AnonymousQueue,AnonymousQueue只允许durable=true,exclusive=false,auto-delete=true if (attributeHasIllegalOverride(element, "durable", "false") || attributeHasIllegalOverride(element, "exclusive", "true") || attributeHasIllegalOverride(element, "auto-delete", "true")) { parserContext.getReaderContext().error( "Anonymous queue cannot specify durable='true', exclusive='false' or auto-delete='false'", element); return; } } else { NamespaceUtils.addConstructorArgBooleanValueIfAttributeDefined(builder, element, "durable", false); NamespaceUtils .addConstructorArgBooleanValueIfAttributeDefined(builder, element, "exclusive", false); NamespaceUtils.addConstructorArgBooleanValueIfAttributeDefined(builder, element, "auto-delete", false); } String queueArguments = element.getAttribute("queue-arguments"); Element argumentsElement = DomUtils.getChildElementByTagName(element, "queue-arguments"); //不允许queue标签和子标签同时存在queue-arguments参数,如下 <rabbit:queue-arguments id="queue_arguments"> <entry key="x-message-ttl" value="30000" value-type="java.lang.Long" /> <entry key="x-dead-letter-exchange" value="trade_direct"/> <entry key="x-dead-letter-routing-key" value="routeKey_TradePayNotify"/> </rabbit:queue-arguments> <rabbit:queue id="queue_TestNotify_delay_15sXXXXXXBB" name="queue_TestNotify_delay_15sXXXXXXBB" queue-arguments="queue_arguments" durable="true" auto-delete="false" exclusive="false"> <rabbit:queue-arguments> <entry key="x-message-ttl" value="30000" value-type="java.lang.Long" /> <entry key="x-dead-letter-exchange" value="trade_direct"/> <entry key="x-dead-letter-routing-key" value="routeKey_TradePayNotify"/> </rabbit:queue-arguments> </rabbit:queue> 情况的出现 if (argumentsElement != null) { if (StringUtils.hasText(queueArguments)) { parserContext .getReaderContext() .error("Queue may have either a queue-attributes attribute or element, but not both", element); } String ref = argumentsElement.getAttribute("ref"); 将entry元素转化为map,如下 <entry key="x-message-ttl" value="30000" value-type="java.lang.Long" /> <entry key="x-dead-letter-exchange" value="trade_direct"/> <entry key="x-dead-letter-routing-key" value="routeKey_TradePayNotify"/> Map<?, ?> map = parserContext.getDelegate().parseMapElement(argumentsElement, builder.getRawBeanDefinition()); if (StringUtils.hasText(ref)) { if (map != null && map.size() > 0) { parserContext.getReaderContext() .error("You cannot have both a 'ref' and a nested map", element); } builder.addConstructorArgReference(ref); } else { builder.addConstructorArgValue(map); } } if (StringUtils.hasText(queueArguments)) { builder.addConstructorArgReference(queueArguments); } NamespaceUtils.parseDeclarationControls(element, builder); } private boolean attributeHasIllegalOverride(Element element, String name, String allowed) { return element.getAttributeNode(name) != null && element.getAttributeNode(name).getSpecified() && !allowed.equals(element.getAttribute(name)); } }
从上述代码中,队列的解析,先对name,id进行较验,再对队列queue-arguments参数进行较验,较验通过后,将name,id,queue-arguments Map参数存储到Queue的BeanDefinition中,当没有为队列设置名称时,Spring会设置BeanClass为AnonymousQueue,那Queue和AnonymousQueue有什么区别呢?先来看看类结构。
从类结构可以看出,AnonymousQueue继承了Queue,区别在于AnonymousQueue的构造方法自动为Queue设置队列名称,并且指定AnonymousQueue的durable默认值为false,exclusive为true,autoDelete为true。也就是说没有指定名称,则队列是非持久化的,如果队列中的消息没有被消费的话,重启系统会丢失,非排他的,同时当最后一个消费者取消订阅的时候,队列就会被自动移除了。从后面的较验参数中得知AnonymousQueue只允许durable=true,exclusive=false,auto-delete=true,没有名称的队列设置或者不设置这三个参数 durable,exclusive,auto-delete都无关紧要,设置反而容易报错。
public abstract class AbstractExchangeParser extends AbstractSingleBeanDefinitionParser { protected void doParse(Element element, ParserContext parserContext, BeanDefinitionBuilder builder) { String exchangeName = element.getAttribute("name"); builder.addConstructorArgValue(new TypedStringValue(exchangeName)); this.parseBindings(element, parserContext, builder, exchangeName); //durable 设置是否持久 durab 设置为 true 表示持久化, 反之是非持久,设置为true则将Exchange存盘,即使服务器重启数据也不会丢失 NamespaceUtils.addConstructorArgBooleanValueIfAttributeDefined(builder, element, "durable", true); //autoDelete 设置是否自动删除,当最后一个绑定到Exchange上的队列删除后,自动删除该Exchange,简单来说也就是如果该Exchange没有和任何队列Queue绑定则删除 NamespaceUtils.addConstructorArgBooleanValueIfAttributeDefined(builder, element, "auto-delete", false); NamespaceUtils.setValueIfAttributeDefined(builder, element, "delayed"); //internal 设置是否是RabbitMQ内部使用,默认false。如果设置为 true ,则表示是内置的交换器,客户端程序无法直接发送消息到这个交换器中,只能通过交换器路由到交换器这种方式。 NamespaceUtils.setValueIfAttributeDefined(builder, element, "internal"); //argument 扩展参数,用于扩展AMQP协议自制定化使用 this.parseArguments(element, "exchange-arguments", parserContext, builder, (String)null); NamespaceUtils.parseDeclarationControls(element, builder); } protected void parseBindings(Element element, ParserContext parserContext, BeanDefinitionBuilder builder, String exchangeName) { Element bindingsElement = DomUtils.getChildElementByTagName(element, "bindings"); this.doParseBindings(element, parserContext, exchangeName, bindingsElement, this); } protected void doParseBindings(Element element, ParserContext parserContext, String exchangeName, Element bindings, AbstractExchangeParser parser) { if (bindings != null) { Iterator var6 = DomUtils.getChildElementsByTagName(bindings, "binding").iterator(); while(var6.hasNext()) { Element binding = (Element)var6.next(); BeanDefinitionBuilder bindingBuilder = parser.parseBinding(exchangeName, binding, parserContext); NamespaceUtils.parseDeclarationControls(element, bindingBuilder); BeanDefinition beanDefinition = bindingBuilder.getBeanDefinition(); this.registerBeanDefinition(new BeanDefinitionHolder(beanDefinition, parserContext.getReaderContext().generateBeanName(beanDefinition)), parserContext.getRegistry()); } } } } public class DirectExchangeParser extends AbstractExchangeParser { private static final String BINDING_KEY_ATTR = "key"; @Override protected Class<?> getBeanClass(Element element) { return DirectExchange.class; } @Override protected BeanDefinitionBuilder parseBinding(String exchangeName, Element binding, ParserContext parserContext) { BeanDefinitionBuilder builder = BeanDefinitionBuilder.genericBeanDefinition(BindingFactoryBean.class); parseDestination(binding, parserContext, builder); builder.addPropertyValue("exchange", new TypedStringValue(exchangeName)); //队列的id String queueId = binding.getAttribute("queue"); //交换器id String exchangeId = binding.getAttribute("exchange"); String bindingKey = binding.hasAttribute("key") ? binding.getAttribute("key") : "#{@'" + (StringUtils.hasText(queueId) ? queueId : exchangeId) + "'.name}"; //绑定的路由键 builder.addPropertyValue("routingKey", new TypedStringValue(bindingKey)); return builder; } }
在上述过程中,省略了FanoutExchangeParser,HeadersExchangeParser,TopicExchangeParser解析器源码的书写,其内部实现上也是大同小异,但是公共部分是在抽象类AbstractExchangeParser对交换器的名称,durable,auto-delete,delayed,internal等参数的解析,不同的部分,在类的内部实现,下面来看一下类的继承结构。
从类的继续结构上来看,FanoutExchangeParser,HeadersExchangeParser,TopicExchangeParser ,DirectExchangeParser他们分别重写了getBeanClass,和parseBinding方法,内部实现了自己的参数绑定。以及BeanClass的指定。
class ListenerContainerParser implements BeanDefinitionParser { @SuppressWarnings("unchecked") @Override public BeanDefinition parse(Element element, ParserContext parserContext) { CompositeComponentDefinition compositeDef = new CompositeComponentDefinition(element.getTagName(), parserContext.extractSource(element)); parserContext.pushContainingComponent(compositeDef); String group = element.getAttribute("group"); ManagedList<RuntimeBeanReference> containerList = null; if (StringUtils.hasText(group)) { BeanDefinition groupDef; if (parserContext.getRegistry().containsBeanDefinition(group)) { groupDef = parserContext.getRegistry().getBeanDefinition(group); } else { BeanDefinitionBuilder builder = BeanDefinitionBuilder.genericBeanDefinition(ArrayList.class); builder.addConstructorArgValue(new ManagedList<RuntimeBeanReference>()); groupDef = builder.getBeanDefinition(); BeanDefinitionHolder holder = new BeanDefinitionHolder(groupDef, group); BeanDefinitionReaderUtils.registerBeanDefinition(holder, parserContext.getRegistry()); } ConstructorArgumentValues constructorArgumentValues = groupDef.getConstructorArgumentValues(); if (!ArrayList.class.getName().equals(groupDef.getBeanClassName()) || constructorArgumentValues.getArgumentCount() != 1 || constructorArgumentValues.getIndexedArgumentValue(0, ManagedList.class) == null) { parserContext.getReaderContext().error("Unexpected configuration for bean " + group, element); } containerList = (ManagedList<RuntimeBeanReference>) constructorArgumentValues .getIndexedArgumentValue(0, ManagedList.class).getValue(); } List<Element> childElements = DomUtils.getChildElementsByTagName(element, "listener"); for (int i = 0; i < childElements.size(); i++) { parseListener(childElements.get(i), element, parserContext, containerList); } parserContext.popAndRegisterContainingComponent(); return null; } }
上述过程中如果配置了组,则添加组的BeanDefinition保存到容器中,如果没有,则对下面这一行进行解析,在parseListener方法中,将设置ref,method,message-converter,response-exchange,添加到MessageListenerAdapter的beanDefinition中,将exclusive,queue-names,queues,x-priority,admin 等添加到SimpleMessageListenerContainer的beanDefinition中。
<rabbit:listener-container connection-factory="connectionFactory" acknowledge="auto" task-executor="" error-handler="" transaction-manager="" concurrency="" max-concurrency="" min-start-interval="" min-stop-interval="" min-consecutive-active="" min-consecutive-idle="" prefetch="" receive-timeout="" channel-transacted="" transaction-size="" requeue-rejected="" phase="" auto-startup="" advice-chain="" recovery-interval="" recovery-back-off="" missing-queues-fatal="" mismatched-queues-fatal="" auto-declare="" declaration-retries="" failed-declaration-retry-interval="" missing-queue-retry-interval="" consumer-tag-strategy="" idle-event-interval=""> <rabbit:listener queues="test_queue_key" ref="queueListenter" response-exchange="" exclusive="" admin="" method="" queue-names=" " priority="" id="" response-routing-key="" /> </rabbit:listener-container>
接下来,我们来看rabbit:template解析过程,先来看看rabbit:template的使用。
<rabbit:template connection-factory="" channel-transacted="" queue="" exchange="" routing-key="" receive-timeout="" reply-timeout="" encoding="" message-converter="" reply-address="" reply-queue="" id="" use-temporary-reply-queues="" return-callback="" confirm-callback="" correlation-key="" recovery-callback="" mandatory="" ></rabbit:template>
那rabbit:template是如何解析的呢?
class TemplateParser extends AbstractSingleBeanDefinitionParser { @Override protected Class<?> getBeanClass(Element element) { return RabbitTemplate.class; } @Override protected boolean shouldGenerateId() { return false; } @Override protected boolean shouldGenerateIdAsFallback() { return false; } @Override protected void doParse(Element element, ParserContext parserContext, BeanDefinitionBuilder builder) { String connectionFactoryRef = element.getAttribute("connection-factory"); if (!StringUtils.hasText(connectionFactoryRef)) { parserContext.getReaderContext().error("A '" + "connection-factory" + "' attribute must be set.", element); } if (StringUtils.hasText(connectionFactoryRef)) { // Use constructor with connectionFactory parameter builder.addConstructorArgReference(connectionFactoryRef); } NamespaceUtils.setValueIfAttributeDefined(builder, element, "channel-transacted"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "queue"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "exchange"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "routing-key"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "receive-timeout"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "reply-timeout"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "encoding"); NamespaceUtils.setReferenceIfAttributeDefined(builder, element, "message-converter"); String replyAddress = element.getAttribute("reply-address"); if (!StringUtils.hasText(replyAddress)) { NamespaceUtils.setReferenceIfAttributeDefined(builder, element, "reply-queue", Conventions.attributeNameToPropertyName("reply-address")); } NamespaceUtils.setValueIfAttributeDefined(builder, element, "use-temporary-reply-queues"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "reply-address"); NamespaceUtils.setReferenceIfAttributeDefined(builder, element, "return-callback"); NamespaceUtils.setReferenceIfAttributeDefined(builder, element, "confirm-callback"); NamespaceUtils.setValueIfAttributeDefined(builder, element, "correlation-key"); NamespaceUtils.setReferenceIfAttributeDefined(builder, element, "retry-template"); NamespaceUtils.setReferenceIfAttributeDefined(builder, element, "recovery-callback"); BeanDefinition expressionDef = NamespaceUtils.createExpressionDefinitionFromValueOrExpression("mandatory", "mandatory-expression", parserContext, element, false); if (expressionDef != null) { builder.addPropertyValue("mandatoryExpression", expressionDef); } BeanDefinition sendConnectionFactorySelectorExpression = NamespaceUtils.createExpressionDefIfAttributeDefined("send-connection-factory-selector-expression", element); if (sendConnectionFactorySelectorExpression != null) { builder.addPropertyValue("sendConnectionFactorySelectorExpression", sendConnectionFactorySelectorExpression); } BeanDefinition receiveConnectionFactorySelectorExpression = NamespaceUtils.createExpressionDefIfAttributeDefined("receive-connection-factory-selector-expression", element); if (receiveConnectionFactorySelectorExpression != null) { builder.addPropertyValue("receiveConnectionFactorySelectorExpression", receiveConnectionFactorySelectorExpression); } BeanDefinition userIdExpression = NamespaceUtils.createExpressionDefIfAttributeDefined("user-id-expression", element); if (userIdExpression != null) { builder.addPropertyValue("userIdExpression", userIdExpression); } BeanDefinition replyContainer = null; Element childElement = null; List<Element> childElements = DomUtils.getChildElementsByTagName(element, "reply-listener"); if (childElements.size() > 0) { childElement = childElements.get(0); } if (childElement != null) { replyContainer = parseListener(childElement, element, parserContext); if (replyContainer != null) { replyContainer.getPropertyValues().add("messageListener", new RuntimeBeanReference(element.getAttribute("id"))); String replyContainerName = element.getAttribute("id") + ".replyListener"; parserContext.getRegistry().registerBeanDefinition(replyContainerName, replyContainer); } } if (replyContainer == null && element.hasAttribute("reply-queue")) { parserContext.getReaderContext().error( "For template '" + element.getAttribute("id") + "', when specifying a reply-queue, " + "a <reply-listener/> element is required", element); } else if (replyContainer != null && !element.hasAttribute("reply-queue")) { parserContext.getReaderContext().error( "For template '" + element.getAttribute("id") + "', a <reply-listener/> element is not allowed if no " + "'reply-queue' is supplied", element); } } private BeanDefinition parseListener(Element childElement, Element element, ParserContext parserContext) { BeanDefinition replyContainer = RabbitNamespaceUtils.parseContainer(childElement, parserContext); if (replyContainer != null) { replyContainer.getPropertyValues().add( "connectionFactory", new RuntimeBeanReference(element.getAttribute("connection-factory"))); } if (element.hasAttribute("reply-queue")) { replyContainer.getPropertyValues().add("queues", new RuntimeBeanReference(element.getAttribute("reply-queue"))); } return replyContainer; } }
上述过程中,也没有太多解释的,就是对rabbit:template中各个参数解析并添加到BeanDefinition中,并设置BeanClass为RabbitTemplate。关于参数的解析,就到这里了,其他的解析方式也大同小异。下面我们来看注册的Bean在项目启动时做了哪些准备工作。
生产者源码解析
从生产者代码来看,最核心的是RabbitTemplate,那我们先来看看RabbitTemplate类结构 。
从bean的生命周期来看,RabbitTemplate实现了InitializingBean接口,必然实现afterPropertiesSet方法,那在afterPropertiesSet方法中做了哪些事情呢?先来看方法。
public void afterPropertiesSet() { Assert.notNull(this.connectionFactory, "ConnectionFactory is required"); }
原来如此,如果没有配置connectionFactory,则实例化bean时,会抛出ConnectionFactory is required异常。
接下来,我们再来看RabbitTemplate实现了BeanFactoryAware接口,而实现这个接口的主要目的是得到BeanFactory,那得到BeanFactory的用处是什么呢?
private final StandardEvaluationContext evaluationContext = new StandardEvaluationContext(); public void setBeanFactory(BeanFactory beanFactory) throws BeansException { this.evaluationContext.setBeanResolver(new BeanFactoryResolver(beanFactory)); this.evaluationContext.addPropertyAccessor(new MapAccessor()); }
从代码中可以看到,Spring将beanFactory加入到StandardEvaluationContext中,那加入到其中的用处又是什么呢?再来看代码。
public void send(final String exchange, final String routingKey, final Message message, final CorrelationData correlationData) throws AmqpException { execute(new ChannelCallback<Object>() { @Override public Object doInRabbit(Channel channel) throws Exception { doSend(channel, exchange, routingKey, message, RabbitTemplate.this.returnCallback != null && RabbitTemplate.this.mandatoryExpression.getValue( RabbitTemplate.this.evaluationContext, message, Boolean.class), correlationData); return null; } }, obtainTargetConnectionFactoryIfNecessary(this.sendConnectionFactorySelectorExpression, message)); }
从发送消息地方来看,通RabbitTemplate.this.returnCallback != null && RabbitTemplate.this.mandatoryExpression.getValue( RabbitTemplate.this.evaluationContext, message, Boolean.class) 来控制mandatory参数,当mandatory标志位设置为true时,如果exchange根据自身类型和消息routeKey无法找到一个符合条件的queue,那么会调用basic.return方法将消息返回给生产者(Basic.Return + Content-Header + Content-Body);当mandatory设置为false时,出现上述情形broker会直接将消息扔掉。因此,在Spring中如果exchange根据自身类型和消息routeKey无法找到一个符合条件的queue,必需配置returnCallback并且mandatory参数为true时,才生效,也就是说RabbitTemplate.this.mandatoryExpression.getValue( RabbitTemplate.this.evaluationContext, message, Boolean.class)方法调用,是为了得到配置文件中 mandatory=“xxxx”,xxxx的boolean值。此时我们再回来看TemplateParser是如何解析mandatory参数的。
BeanDefinition expressionDef = NamespaceUtils.createExpressionDefinitionFromValueOrExpression("mandatory", "mandatory-expression", parserContext, element, false); public static BeanDefinition createExpressionDefinitionFromValueOrExpression(String valueElementName, String expressionElementName, ParserContext parserContext, Element element, boolean oneRequired) { Assert.hasText(valueElementName, "'valueElementName' must not be empty"); Assert.hasText(expressionElementName, "'expressionElementName' must not be empty"); String valueElementValue = element.getAttribute(valueElementName); String expressionElementValue = element.getAttribute(expressionElementName); boolean hasAttributeValue = StringUtils.hasText(valueElementValue); boolean hasAttributeExpression = StringUtils.hasText(expressionElementValue); if (hasAttributeValue && hasAttributeExpression) { parserContext.getReaderContext().error("Only one of '" + valueElementName + "' or '" + expressionElementName + "' is allowed", element); } if (oneRequired && (!hasAttributeValue && !hasAttributeExpression)) { parserContext.getReaderContext().error("One of '" + valueElementName + "' or '" + expressionElementName + "' is required", element); } BeanDefinition expressionDef; if (hasAttributeValue) { expressionDef = new RootBeanDefinition(LiteralExpression.class); expressionDef.getConstructorArgumentValues().addGenericArgumentValue(valueElementValue); } else { expressionDef = createExpressionDefIfAttributeDefined(expressionElementName, element); } return expressionDef; }
从上述中,我们可以看到,当rabbit:template中配置了 mandatory="xxx"参数,则在RabbitTemplate中对应的mandatoryExpression属性是LiteralExpression对象,并将xxx参数设置到LiteralExpression的literalValue属性中,否则,则使用我们自定义的mandatory-expression="yyy"中定义的bean来填充RabbitTemplate中的mandatoryExpression属性。那我们再回头来看下面代码。
RabbitTemplate.this.mandatoryExpression.getValue(
RabbitTemplate.this.evaluationContext, message, Boolean.class)
如果配置了mandatory参数,则此时mandatoryExpression为LiteralExpression对象,那进入getValue方法。
public String getValue(EvaluationContext context, Object rootObject) throws EvaluationException { return this.literalValue; } @Override public <T> T getValue(EvaluationContext context, Object rootObject, Class<T> desiredResultType) throws EvaluationException { Object value = getValue(context, rootObject); return ExpressionUtils.convertTypedValue(context, new TypedValue(value), desiredResultType); }
从上述中可以看出,通过getValue方法获得mandatory=“xxx”,xxx值,再通过ExpressionUtils.convertTypedValue()方法,将xxx转化为desiredResultType类型,而我们知道在RabbitTemplate中调用getValue方法传入的是Boolean.class,因此最终得到mandatory配置的boolean值。但是细心的读者有没有发现整个过程EvaluationContext都没有用到,那EvaluationContext在什么场景下使用呢?我们先来看一个例子。
- 创建return回调监听器
public class MQReturnListener implements RabbitTemplate.ReturnCallback { @Override public void returnedMessage(Message message, int replyCode, String replyText, String exchange, String routingKey) { System.out.println("message = " + message); System.out.println("replyCode = " + replyCode + ",replyText="+replyText + ",exchange="+exchange + ",routingKey="+routingKey); } }
- 配置Spring配置文件
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:rabbit="http://www.springframework.org/schema/rabbit" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.2.xsd http://www.springframework.org/schema/rabbit http://www.springframework.org/schema/rabbit/spring-rabbit-1.6.xsd"> <!--spring.rabbitmq.publisher-confirms=true 设置此属性配置可以确保消息成功发送到交换器 , 1.publiser-confirm模式可以确保生产者到交换器exchange消息有没有发送成功--> <!--spring.rabbitmq.publisher-returns=true 可以确保消息在未被队列接收时返回 , 2.publisher-return模式可以在消息没有被路由到指定的queue时将消息返回,而不是丢弃--> <!-- spring.rabbitmq.template.mandatory=true , 指定消息在没有被队列接收时是否强行退回还是直接丢弃 , 在使用上面的属性配置时通常会和mandatory属性配合一起使用:--> <rabbit:connection-factory id="connectionFactory" host="172.16.157.242" username="guest" password="guest" port="5672" virtual-host="/" publisher-returns="true"/> <rabbit:admin connection-factory="connectionFactory"/> <!--申明一个消息队列Queue durable:是否持久化 exclusive: 仅创建者可以使用的私有队列,断开后自动删除 auto_delete: 当所有消费客户端连接断开后,是否自动删除队列--> <rabbit:queue id="test_queue_key" name="test_queue_key" durable="true" auto-delete="false" exclusive="false"/> <rabbit:queue id="queue_TestNotify" name="queue_TestNotify" durable="true" auto-delete="false" exclusive="false"> <rabbit:queue-arguments> <entry key="x-dead-letter-exchange" value="trade_direct"/> <entry key="x-dead-letter-routing-key" value="routeKey_TradePayNotify_delay_15s"/> </rabbit:queue-arguments> </rabbit:queue> <!--<rabbit:queue-arguments id="queue_arguments"> <entry key="x-message-ttl" value="30000" value-type="java.lang.Long" /> <entry key="x-dead-letter-exchange" value="trade_direct"/> <entry key="x-dead-letter-routing-key" value="routeKey_TradePayNotify"/> </rabbit:queue-arguments>--> <!--<rabbit:queue id="queue_TestNotify_delay_15sXXXXXXBB" name="queue_TestNotify_delay_15sXXXXXXBB" queue-arguments="queue_arguments" durable="true" auto-delete="false" exclusive="false">--> <rabbit:queue id="queue_TestNotify_delay_15sXXXXXXBB" name="queue_TestNotify_delay_15sXXXXXXBB" durable="true" auto-delete="false" exclusive="false"> <rabbit:queue-arguments> <entry key="x-message-ttl" value="30000" value-type="java.lang.Long" /> <entry key="x-dead-letter-exchange" value="trade_direct"/> <entry key="x-dead-letter-routing-key" value="routeKey_TradePayNotify"/> </rabbit:queue-arguments> </rabbit:queue> <!-- work exchange --> <rabbit:direct-exchange name="trade_direct" durable="true" auto-delete="false"> <rabbit:bindings> <rabbit:binding queue="queue_TestNotify" key="routeKey_TradePayNotify"/> <rabbit:binding queue="queue_TestNotify_delay_15sXXXXXXBB" key="routeKey_TradePayNotify_delay_15s"/> </rabbit:bindings> </rabbit:direct-exchange> <!-- 声明Exchange rabbit:direct-exchange:定义exchange模式为direct --> <rabbit:direct-exchange name="test-mq-exchangeaaaaa" durable="true" auto-delete="false" id="test-mq-exchangeaaaaa"> <!-- <rabbit:bindings> <rabbit:binding queue="test_queue_key" key="test_queue_key"/> </rabbit:bindings>--> </rabbit:direct-exchange> <bean id="mqConfirmListener" class="com.spring_1_100.test_71_80.test78_spring_rabbitmq.spring3_mandatory_expression.MQReturnListener"></bean> <!-- spring template声明--> <rabbit:template id="rabbitTemplate" exchange="test-mq-exchangeaaaaa" connection-factory="connectionFactory" mandatory-expression="body.length > 10" message-converter="jsonMessageConverter" return-callback="mqConfirmListener"> </rabbit:template> <!-- 消息对象json转换类 --> <bean id="jsonMessageConverter" class="org.springframework.amqp.support.converter.Jackson2JsonMessageConverter"/> </beans>
上述加粗的字体,大家可能会觉得莫名其妙,为什么加body.length > 10 呢?这个是什么意思呢?在看到测试结果再来分析。
- 开始测试
public class MQSender { public static final String queue_key = "test_queue_key"; public static void main(String[] args) { ApplicationContext context = new ClassPathXmlApplicationContext("classpath:spring_1_100/config_71_80/spring78_rabbitmq/spring3/spring78_mq_sender3.xml"); Map<String, Object> msg = new HashMap(); msg.put("data", "hello,rabbmitmq!"); msg.put("time", System.currentTimeMillis()); System.out.println("++++++++发送消息++++++++++++" + JSON.toJSONString(msg)); RabbitTemplate rabbitTemplate = (RabbitTemplate) context.getBean("rabbitTemplate"); rabbitTemplate.convertAndSend(queue_key, msg); } }
开始测试:
测试效果如下所示,同样出现了消息回吐,那么mandatory-expression="body.length > 10"到底起到作用了吗?不知道,那么我们将mandatory-expression="body.length > 100"试试
从上述效果来看,发现奇际出来了,竟然没有出现消息回吐,证明我们的配置mandatory-expression="body.length > 100"起到了作用,为什么呢? 我们继续来看源码。先来看看mandatory-expression标签的解析,代码如下。
public static BeanDefinition createExpressionDefIfAttributeDefined(String expressionElementName, Element element) { Assert.hasText(expressionElementName, "'expressionElementName' must no be empty"); String expressionElementValue = element.getAttribute(expressionElementName); if (StringUtils.hasText(expressionElementValue)) { BeanDefinitionBuilder expressionDefBuilder = BeanDefinitionBuilder.genericBeanDefinition(ExpressionFactoryBean.class); expressionDefBuilder.addConstructorArgValue(expressionElementValue); return expressionDefBuilder.getRawBeanDefinition(); } return null; }
从上述源码中来看,创建了ExpressionFactoryBean的BeanDefinition,并将mandatory-expression="xxx"的参数xxx添加到,但是有没有小伙伴觉得奇怪,我们配置mandatoryExpression参数的bean的定义好像是一个FactoryBean,也就是说实际注入到RabbitTemplate的mandatoryExpression属性值不是ExpressionFactoryBean本身实例,而是ExpressionFactoryBean的getObject方法返回对象值。那我们先来看ExpressionFactoryBean的结构 。
正如所料,因为ExpresionFactoryBean实现了FactoryBean接口,因此,实际注入的属性是其getObject方法返回的对象,来看看getObject方法。
我们最终得到RabbitTemplate的mandatoryExpression属性实际上是由ExpresionFactoryBean的createInstance()方法返回值来填充的,那createInstance()方法又是如何来实现的呢?
private final static ExpressionParser DEFAULT_PARSER = new SpelExpressionParser(); private final String expressionString; private volatile ExpressionParser parser = DEFAULT_PARSER; protected Expression createInstance() throws Exception { return this.parser.parseExpression(this.expressionString); }
从上述代码中可以看出,最终是调用了this.parser.parseExpression(“body.length > 100”);方法返回的值填充mandatoryExpression属性,而parser的默认值又是SpelExpressionParser对象,当然返回的亦是SpelExpression对象了,关于body.length > 100内部是如何解析的,在之前的博客中也详细说明过,如果要说明这行表达式的解析,一篇博客都无法讲清楚的,感兴趣的小伙伴可以去看我之前的博客。我们知道mandatoryExpression最终的值是SpelExpression表达式,那么他又是如何得到mandatory的boolean值的呢?
从上图得知RabbitTemplate.this.mandatoryExpression.getValue(
RabbitTemplate.this.evaluationContext, message, Boolean.class)的getValue方法,实际上是调用了SpelExpression的getValue方法,getTypeValue方法调用实际上是判断Message的body属性length长度是否大于 10 或者 大于100,从而得到boolean值,而在getTypeValue的内部实际上也是通过反射等技术来实现的,但是实际情况也非常复杂,而内部肯定需要用到EvaluationContext中的BeanFactory,而实现过程这里就不再赘述,当然上面也两中情况也只是一个示例,还可以按下面这种写法 ,如mandatory-expression=“messageProperties.contentType.equals(‘application/jsonxxx’)” 这里我也只抛砖引玉一下,更多的应用场景不需要读者根据业务具体情况去设置。,在这个示例中,我们得到了两个结论,第一,如果配置mandatory-expression参数,参数的作用对象是Message,mandatory-expression参数的解析肯定要用到BeanFactory,因此RabbitTemplate需要实现BeanFactoryAware接口。我们写了这么多,只为说明RabbitTemplate继承BeanFactoryAware的作用,Spring太博大精深了,即使艰难,但是研究还得继续,我们接着来研究RabbitTemplate发送消息这一块。
public void send(final String exchange, final String routingKey, final Message message, final CorrelationData correlationData) throws AmqpException { execute(new ChannelCallback<Object>() { @Override public Object doInRabbit(Channel channel) throws Exception { doSend(channel, exchange, routingKey, message, RabbitTemplate.this.returnCallback != null && RabbitTemplate.this.mandatoryExpression.getValue( RabbitTemplate.this.evaluationContext, message, Boolean.class), correlationData); return null; } }, obtainTargetConnectionFactoryIfNecessary(this.sendConnectionFactorySelectorExpression, message)); }
private <T> T execute(final ChannelCallback<T> action, final ConnectionFactory connectionFactory) { if (this.retryTemplate != null) { try { return this.retryTemplate.execute(new RetryCallback<T, Exception>() { @Override public T doWithRetry(RetryContext context) throws Exception { return RabbitTemplate.this.doExecute(action, connectionFactory); } }, (RecoveryCallback<T>) this.recoveryCallback); } catch (Exception e) { if (e instanceof RuntimeException) { throw (RuntimeException) e; } throw RabbitExceptionTranslator.convertRabbitAccessException(e); } } else { return doExecute(action, connectionFactory); } }
在上面代码中我们看到了retryTemplate的使用,这个是用来做什么用的呢?从字面意思上来说,重新尝试,一般是在调用doWithRetry方法时,抛出异常,则进行重新尝试,为了体现效果,我们来看一个例子。
1.创建配置文件
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:rabbit="http://www.springframework.org/schema/rabbit" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd http://www.springframework.org/schema/rabbit http://www.springframework.org/schema/rabbit/spring-rabbit-1.4.xsd"> <rabbit:connection-factory id="connectionFactory" host="172.16.157.248" username="guest" password="guest" port="5672" virtual-host="/" publisher-confirms="true"/> <bean id="mqConfirmListener" class="com.spring_1_100.test_71_80.test78_spring_rabbitmq.spring4_retry_template.MQConfirmListener"></bean> <bean id="simpleRetryPolicy" class="org.springframework.retry.policy.SimpleRetryPolicy"> <!--最大尝试次数--> <property name="maxAttempts" value="5"></property> </bean> <!-- 指数退避策略 等待时间= 等待时间*倍数 ,即每一次的等待时间是上一次等待时间的n倍, 到达最大的等待时间之后就不在增加了,一直都是以最大的等待时间在等待。默认执行3次--> <bean id="exponentialBackOffPolicy" class="org.springframework.retry.backoff.ExponentialBackOffPolicy"> <!--初始等待时间--> <property name="initialInterval" value="1000"></property> <!--时间等待倍数--> <property name="multiplier" value="2"></property> <!--最大等待时间--> <property name="maxInterval" value="10000"></property> </bean> <bean id="retryTemplate" class="com.spring_1_100.test_71_80.test78_spring_rabbitmq.spring4_retry_template.MyRetryTemplate"> <property name="backOffPolicy" ref="exponentialBackOffPolicy"></property> <property name="retryPolicy" ref="simpleRetryPolicy"></property> </bean> <!-- spring template声明--> <rabbit:template id="rabbitTemplate" exchange="test-mq-exchangexxxxxbbb" connection-factory="connectionFactory" mandatory="true" message-converter="jsonMessageConverter" confirm-callback="mqConfirmListener" retry-template="retryTemplate" > </rabbit:template> <!-- 消息对象json转换类 --> <bean id="jsonMessageConverter" class="org.springframework.amqp.support.converter.Jackson2JsonMessageConverter"/> </beans>
为了体现测试效果,我们重写retryTemplate,让其在调用过程中抛出异常。
2.重写retryTemplate类
public class MyRetryTemplate extends RetryTemplate { protected <T, E extends Throwable> T doExecute(RetryCallback<T, E> retryCallback, RecoveryCallback<T> recoveryCallback, RetryState state) throws E, ExhaustedRetryException { RetryCallback myRetryCallback = new RetryCallback<String, Throwable>() { @Override public String doWithRetry(RetryContext context) throws Throwable { //需要重试的代码 System.out.println("开始执行======"); throw new TimeoutException(); } }; RecoveryCallback myRecoveryCallback = new RecoveryCallback<String>() { @Override public String recover(RetryContext context) throws Exception { //重试失败后执行的代码 System.out.println("failed callback======"); return "failed callback"; } }; return (T) super.doExecute(myRetryCallback, myRecoveryCallback, state); } }
- 开始测试
public class MQSender { public static final String queue_key = "test_queue_keybbbbbbbxxxxxx"; public static void main(String[] args) { ApplicationContext context = new ClassPathXmlApplicationContext("classpath:spring_1_100/config_71_80/spring78_rabbitmq/spring78_mq_sender4.xml"); Map<String, Object> msg = new HashMap(); msg.put("data", "hello,rabbmitmq!"); msg.put("time", System.currentTimeMillis()); System.out.println("++++++++发送消息++++++++++++" + JSON.toJSONString(msg)); RabbitTemplate rabbitTemplate = (RabbitTemplate) context.getBean("rabbitTemplate"); rabbitTemplate.convertAndSend(queue_key, msg); } }
【测试结果】
测试结果正如所料,最多请求5次,每一次请求的等待时间是上一次的两倍。那么我们来看看retryTemplate的重试源码实现。
protected <T, E extends Throwable> T doExecute(RetryCallback<T, E> retryCallback, RecoveryCallback<T> recoveryCallback, RetryState state) throws E, ExhaustedRetryException { RetryPolicy retryPolicy = this.retryPolicy; BackOffPolicy backOffPolicy = this.backOffPolicy; //根据策略创建重试上下文 RetryContext context = open(retryPolicy, state); if (logger.isTraceEnabled()) { logger.trace("RetryContext retrieved: " + context); } //将上下文信息保存到threadLocal,用于重试过程中的数据存储,如重试次数,最后一次抛出的异常 RetrySynchronizationManager.register(context); Throwable lastException = null; try { //如果注册了监听器,会调用监听器对应的open方法,如果返回false,则终止执行 boolean running = doOpenInterceptors(retryCallback, context); if (!running) { throw new TerminatedRetryException( "Retry terminated abnormally by interceptor before first attempt"); } //设置退避(BackOff)策略,当操作执行失败时, //根据设置的重试策略进行重试。通过BackoffPolicy可以设定再次重试的时间间隔。 BackOffContext backOffContext = null; Object resource = context.getAttribute("backOffContext"); if (resource instanceof BackOffContext) { backOffContext = (BackOffContext) resource; } if (backOffContext == null) { backOffContext = backOffPolicy.start(context); if (backOffContext != null) { context.setAttribute("backOffContext", backOffContext); } } //判断能否重试 while (canRetry(retryPolicy, context) && !context.isExhaustedOnly()) { try { if (logger.isDebugEnabled()) { logger.debug("Retry: count=" + context.getRetryCount()); } //置空上一次重试异常信息 lastException = null; //执行要执行的内容 return retryCallback.doWithRetry(context); } catch (Throwable e) { lastException = e; //当注册了监听器时,调用监听器的onError方法 doOnErrorInterceptors(retryCallback, context, e); try { //根据重试策略更新context内的数据 registerThrowable(retryPolicy, state, context, e); } catch (Exception ex) { throw new TerminatedRetryException("Could not register throwable", ex); } //重试策略根据保存的context中的数据判断能否再次执行 if (canRetry(retryPolicy, context) && !context.isExhaustedOnly()) { try { //根据退避策略等待相应的时间 backOffPolicy.backOff(backOffContext); } catch (BackOffInterruptedException ex) { lastException = e; // back off was prevented by another thread - fail the retry if (logger.isDebugEnabled()) { logger.debug("Abort retry because interrupted: count=" + context.getRetryCount()); } throw ex; } } if (logger.isDebugEnabled()) { logger.debug("Checking for rethrow: count=" + context.getRetryCount()); } //是否终止重试 if (shouldRethrow(retryPolicy, context, state)) { if (logger.isDebugEnabled()) { logger.debug("Rethrow in retry for policy: count=" + context.getRetryCount()); } throw RetryTemplate.<E>wrapIfNecessary(e); } } } if (logger.isDebugEnabled()) { logger.debug("Retry failed last attempt: count=" + context.getRetryCount()); } if (context.isExhaustedOnly()) { rethrow(context, "Retry exhausted after last attempt with no recovery path."); } //1.retryContextCache中移除state //2.调用recoveryCallback的recover方法 //3.抛出异常 return handleRetryExhausted(recoveryCallback, context, state); } catch (Throwable e) { throw RetryTemplate.<E>wrapIfNecessary(e); } finally { //调用策略的close方法 close(retryPolicy, context, state, lastException == null); //调用监听器的close方法 doCloseInterceptors(retryCallback, context, lastException); //清空threadLocal中的RetryContext数据 RetrySynchronizationManager.clear(); } }
上述过程中,代码写了很多,但是逻辑还是很清淅的
- 调用open方法根据策略获取重试上下文
- 调用register方法存储重试上下文保存到ThreadLocal
- 调用监听器的open方法
- 将退避(BackOff)策略保存到重试上下文中
- 判断能否重试
- 执行方法体
- 如果抛出异常,调用监听器的onError方法
- 执行方法体抛出异常保存信息到重试上下文
- 判断能否重试
- 根据退避策略等待相应的时间
- 判断是否终止重试
- 终止重试,调用策略的close方法,调用监听器的close方法,清空threadLocal中的RetryContext数据
上述过程中,重试策略定义了当操作失败时如何进行重试操作,感觉是一个抽象的概念。那么我们来看看Spring中有哪些重试策略。
重试策略
SimpleRetryPolicy 策略
该策略定义了对指定的异常进行若干次重试。默认情况下,对Exception异常及其子类重试3次。如果创建SimpleRetryPolicy并指定重试异常map,可以选择性重试或不进行重试
NeverRetryPolicy 策略
执行一次待执行操作,若出现异常后不进行重试。
AlwaysRetryPolicy 策略
异常后一直重试直到成功。
TimeoutRetryPolicy 策略
在执行execute方法时从open操作开始到调用TimeoutRetryPolicy的canRetry方法这之间所经过的时间。这段时间未超过TimeoutRetryPolicy定义的超时时间,那么执行操作,否则抛出异常。
ExceptionClassifierRetryPolicy 策略
根据产生的异常选择重试策略。
CompositeRetryPolicy 策略
用户指定一组策略,随后根据optimistic选项来确认如何重试。
大家可能还是对策略这一块有点晕,那我们自己来重写一个策略,看一下效果。
- 先定义一个重试策略
public class MySimpleRetryPolicy implements RetryPolicy { private int maxCount = 5; @Override public boolean canRetry(RetryContext context) { MySimpleRetryContext mySimpleRetryContext = (MySimpleRetryContext) context; if(mySimpleRetryContext.getRetryCount() > maxCount){ return false; } for(Throwable throwable : mySimpleRetryContext.getThrowables()){ if(throwable instanceof NullPointerException){ return false; } if(throwable instanceof TimeoutException){ return true; } } return true; } @Override public RetryContext open(RetryContext parent) { return new MySimpleRetryContext(parent); } @Override public void close(RetryContext context) { } @Override public void registerThrowable(RetryContext context, Throwable throwable) { MySimpleRetryContext mySimpleRetryContext = (MySimpleRetryContext) context; mySimpleRetryContext.addThrowables(throwable); mySimpleRetryContext.registerThrowable(throwable); } public class MySimpleRetryContext extends RetryContextSupport { private Set<Throwable> throwables = new HashSet<>(); public MySimpleRetryContext(RetryContext parent) { super(parent); } public Set<Throwable> getThrowables() { return throwables; } public void addThrowables(Throwable throwable) { this.throwables.add(throwable); } } }
这个重试策略的主要功能是,当重试超过5次,则终止重试,如果抛出空指针异常,停止重试,如果是超时异常,则继续重试。
public class MySimpleRetryPolicyTest { public static void main(String[] args) throws Exception,Throwable { RetryTemplate retryTemplate = new RetryTemplate(); //指数退避策略 等待时间= 等待时间*倍数 ,即每一次的等待时间是上一次等待时间的n倍, //到达最大的等待时间之后就不在增加了,一直都是以最大的等待时间在等待。默认执行3次 ExponentialBackOffPolicy exponentialBackOffPolicy = new ExponentialBackOffPolicy(); //初始等待时间 exponentialBackOffPolicy.setInitialInterval(1000); //时间等待倍数 exponentialBackOffPolicy.setMultiplier(2); //最大等待时间 exponentialBackOffPolicy.setMaxInterval(5000); retryTemplate.setBackOffPolicy(exponentialBackOffPolicy); retryTemplate.setRetryPolicy(new MySimpleRetryPolicy()); String execute = retryTemplate.execute(new RetryCallback<String, Throwable>() { @Override public String doWithRetry(RetryContext context) throws Throwable { if(context.getRetryCount() > 2 ){ //需要重试的代码 System.out.println("开始执行NullPointerException======" + context.getRetryCount()); throw new NullPointerException(); } //需要重试的代码 System.out.println("开始执行TimeoutException======" + context.getRetryCount()); throw new TimeoutException(); } }, new RecoveryCallback<String>() { @Override public String recover(RetryContext context) throws Exception { //重试失败后执行的代码 return "failed callback"; } }); System.out.println(execute); } }
【执行结果】
上述过程中,我们对Spring的重试策略及源码都做了详细分析,为了保证消息的可达,可以根据不同的业务选择不同的重试策略,这也是RabbitTemplate支持retryTemplate的意图吧。
private <T> T doExecute(ChannelCallback<T> action, ConnectionFactory connectionFactory) { Assert.notNull(action, "Callback object must not be null"); //获取Rabbit资源持有者 RabbitResourceHolder resourceHolder = ConnectionFactoryUtils.getTransactionalResourceHolder( (connectionFactory != null ? connectionFactory : getConnectionFactory()), isChannelTransacted()); //获取信道 Channel channel = resourceHolder.getChannel(); //如果配置了confirm-callback=""或 return-callback="",则confirmsOrReturnsCapable为true if (this.confirmsOrReturnsCapable == null) { if (getConnectionFactory() instanceof PublisherCallbackChannelConnectionFactory) { PublisherCallbackChannelConnectionFactory pcccf = (PublisherCallbackChannelConnectionFactory) getConnectionFactory(); this.confirmsOrReturnsCapable = pcccf.isPublisherConfirms() || pcccf.isPublisherReturns(); } else { this.confirmsOrReturnsCapable = Boolean.FALSE; } } //为returnCallback或comfirmCallBack添加浏览器 if (this.confirmsOrReturnsCapable) { addListener(channel); } try { if (logger.isDebugEnabled()) { logger.debug("Executing callback on RabbitMQ Channel: " + channel); } //调用Rabbit的发布消息方法 return action.doInRabbit(channel); } catch (Exception ex) { //如果存在事务,但抛出异常,则回滚 if (isChannelLocallyTransacted(channel)) { resourceHolder.rollbackAll(); } throw convertRabbitAccessException(ex); } finally { //释放信道及链接资源 ConnectionFactoryUtils.releaseResources(resourceHolder); } }
上述代码逻辑还是很清晰的
- 获取Rabbit资源持有者
- 为confirm-callback或return-callback添加事件监听器
- 调用发送消息具体实现
- 如果发送消息过程中抛出异常并且配置了事务,则回滚消息
- 释放connection及channel资源
1. 获取Rabbit资源持有者
public static RabbitResourceHolder getTransactionalResourceHolder(final ConnectionFactory connectionFactory, final boolean synchedLocalTransactionAllowed) { return doGetTransactionalResourceHolder(connectionFactory, new ResourceFactory() { @Override public Channel getChannel(RabbitResourceHolder holder) { return holder.getChannel(); } @Override public Connection getConnection(RabbitResourceHolder holder) { return holder.getConnection(); } @Override public Connection createConnection() throws IOException { return connectionFactory.createConnection(); } @Override public Channel createChannel(Connection con) throws IOException { return con.createChannel(synchedLocalTransactionAllowed); } @Override public boolean isSynchedLocalTransactionAllowed() { return synchedLocalTransactionAllowed; } }); }
private static RabbitResourceHolder doGetTransactionalResourceHolder(ConnectionFactory connectionFactory, ResourceFactory resourceFactory) { Assert.notNull(connectionFactory, "ConnectionFactory must not be null"); Assert.notNull(resourceFactory, "ResourceFactory must not be null"); RabbitResourceHolder resourceHolder = (RabbitResourceHolder) TransactionSynchronizationManager .getResource(connectionFactory); if (resourceHolder != null) { Channel channel = resourceFactory.getChannel(resourceHolder); if (channel != null) { return resourceHolder; } } RabbitResourceHolder resourceHolderToUse = resourceHolder; if (resourceHolderToUse == null) { resourceHolderToUse = new RabbitResourceHolder(); } Connection connection = resourceFactory.getConnection(resourceHolderToUse); //NOSONAR Channel channel = null; try { channel = ConsumerChannelRegistry.getConsumerChannel(connectionFactory); if (channel == null && connection == null) { connection = resourceFactory.createConnection(); resourceHolderToUse.addConnection(connection); } if (channel == null) { channel = resourceFactory.createChannel(connection); } resourceHolderToUse.addChannel(channel, connection); if (resourceHolderToUse != resourceHolder) { bindResourceToTransaction(resourceHolderToUse, connectionFactory, resourceFactory.isSynchedLocalTransactionAllowed()); } return resourceHolderToUse; } catch (IOException ex) { RabbitUtils.closeConnection(connection); throw new AmqpIOException(ex); } }
获取RabbitResourceHolder时,如果有事务,则从缓存的resources中获取资源持有都,如果没有事务,则直接New一个RabbitResourceHolder,通过RabbitResourceHolder获取其connection和channel,并分别将connection和channel存到RabbitResourceHolder对象的connections和channels属性中。而connection的实际创建方法实际上是调用了connectionFactory的createConnection和createChannel方法。接下来,我们先来看createConnection方法的内部实现。
createConnection()方法创建过程中提供了两种模式,如下图所示。
CachingConnectionFactory为我们提供了两种缓存的模式:
-
模式一:只缓存Connection
这也是CachingConnectionFactory的默认模式,在这种模式下,所有的createConnection()方法实际上返回的都是同一个Connection,同样的Connection.close()方法是没用的,因为就一个,默认情况下,Connection中只缓存了一个Channel,在并发量不大的时候这种模式是完全够用的,当并发量较高的时候,我们可以setChannelCacheSize()来增加Connection中缓存的Channel的数量。 -
模式二:Connection和Channel都缓存
在CONNECTION模式下,每一次调用createConnection()方法都会新建一个或者从缓存中获取,根据你设置的ConnectionCacheSize的大小,当小于的时候会采用新建的策略,当大于等于的时候会采用从缓存中获取的策略,与CHANNEL模式不同的是,CONNECTION模式对Connection和Channel都进行了缓存,最新版本的client中已经将Channel的缓存数量从1增加到了25,但是在并发量不是特别大的情况下,作用并不是特别明显。使用CachingConnectionFactory需要注意的一点是:所有你获取的Channel对象必须要显式的关闭,所以finally中一定不要忘记释放资源,如果忘记释放,则可能造成连接池中没有资源可用。
public final Connection createConnection() throws AmqpException { Assert.state(!this.stopped, "The ApplicationContext is closed and the ConnectionFactory can no longer create connections."); synchronized (this.connectionMonitor) { // CHANNEL模式下,这里的connection是ChannelCachingConnectionProxy 代理对象 //这样做的目的是为Channel提供临时的存储空间(也就是缓存Channel),以便其他客户端调用 if (this.cacheMode == CacheMode.CHANNEL) { // 确保Connection对象不为null,target是真实的连接 if (this.connection.target == null) { //第一次调用 createConnection 方法时 connection.target 值为 null,因此会调用 createBareConnection 方法创建出 SimpleConnection 赋值给 connection.target //SimpleConnection 中delegate属性是真正的RabbitMQ 连接(AMQConnection) this.connection.target = super.createBareConnection(); if (!this.checkoutPermits.containsKey(this.connection)) { // Map<Connection, Semaphore> checkoutPermits 中存放了信道的许可数量,也就是默认的25,通过信号量来同步资源 this.checkoutPermits.put(this.connection, new Semaphore(this.channelCacheSize)); } // 向所有 ConnectionListener 发布 onCreate 事件 getConnectionListener().onCreate(this.connection); } return this.connection; } else if (this.cacheMode == CacheMode.CONNECTION) { //从空闲的连接池中获取链接 ChannelCachingConnectionProxy connection = findIdleConnection(); long now = System.currentTimeMillis(); //如果配置了channelCheckoutTimeout参数,同时空闲连接为空,则等待channelCheckoutTimeout 毫秒后,再获取空闲链接 while (connection == null && System.currentTimeMillis() - now < this.channelCheckoutTimeout) { if (countOpenConnections() >= this.connectionLimit) { try { // 等待channelCheckoutTimeout毫秒值 this.connectionMonitor.wait(this.channelCheckoutTimeout); connection = findIdleConnection(); } catch (InterruptedException e) { Thread.currentThread().interrupt(); throw new AmqpException("Interrupted while waiting for a connection", e); } } } if (connection == null) { //如果空闲链接数据大于配置的connectionLimit数【默认值是0x7fffffff】,则抛出异常 if (countOpenConnections() >= this.connectionLimit && System.currentTimeMillis() - now >= this.channelCheckoutTimeout) { throw new AmqpTimeoutException("Timed out attempting to get a connection"); } //创建connection代理,当close connection时,实际上是调用ChannelCachingConnectionProxy的close方法 connection = new ChannelCachingConnectionProxy(super.createBareConnection()); if (logger.isDebugEnabled()) { logger.debug("Adding new connection '" + connection + "'"); } this.allocatedConnections.add(connection); this.allocatedConnectionNonTransactionalChannels.put(connection, new LinkedList<ChannelProxy>()); this.channelHighWaterMarks.put(ObjectUtils.getIdentityHexString( this.allocatedConnectionNonTransactionalChannels.get(connection)), new AtomicInteger()); this.allocatedConnectionTransactionalChannels.put(connection, new LinkedList<ChannelProxy>()); this.channelHighWaterMarks.put( ObjectUtils.getIdentityHexString(this.allocatedConnectionTransactionalChannels.get(connection)), new AtomicInteger()); //设置每一个连接允许的最大信道数,默认为25 this.checkoutPermits.put(connection, new Semaphore(this.channelCacheSize)); // 向所有 ConnectionListener 发布 onCreate 事件 getConnectionListener().onCreate(connection); } else if (!connection.isOpen()) { try { // 如果connection 没有open,则关闭当前connection,创建一个新的connection refreshProxyConnection(connection); } catch (Exception e) { this.idleConnections.addLast(connection); } } else { if (logger.isDebugEnabled()) { logger.debug("Obtained connection '" + connection + "' from cache"); } } return connection; } } return null; }
无论是哪种模式下,当connection不存在时,都需要创建connection,下面我们来看connection的创建逻辑。
protected final Connection createBareConnection() { try { Connection connection; if (this.addresses != null) { connection = new SimpleConnection(this.rabbitConnectionFactory.newConnection(this.executorService, this.addresses), this.closeTimeout); } else { connection = new SimpleConnection(this.rabbitConnectionFactory.newConnection(this.executorService), this.closeTimeout); } if (this.logger.isInfoEnabled()) { this.logger.info("Created new connection: " + connection); } return connection; } catch (IOException e) { throw RabbitExceptionTranslator.convertRabbitAccessException(e); } catch (TimeoutException e) { throw RabbitExceptionTranslator.convertRabbitAccessException(e); } }
从上面的代码来看,无论是哪种情况都是创建了一个SimpleConnection对象,其内部delegate中保存了真实的connection对象。我们先记记住目前的connection对象不是原生的rabbitmq的connection对象,而是connection对象的包装类SimpleConnection,真实的connection是SimpleConnection的delegate属性。到底有什么用,在后面的逻辑中,再来分析。
cacheMode有两中模式,分别是CHANNEL和CONNECTION,CHANNEL模式比较简单,这里就不分析了,在CONNECTION模式下,需要注意两个方法,第一个方法是findIdleConnection(),这个方法有什么用呢?
private ChannelCachingConnectionProxy findIdleConnection() { ChannelCachingConnectionProxy connection = null; ChannelCachingConnectionProxy lastIdle = this.idleConnections.peekLast(); while (connection == null) { connection = this.idleConnections.poll(); if (connection != null) { if (!connection.isOpen()) { if (logger.isDebugEnabled()) { logger.debug("Skipping closed connection '" + connection + "'"); } connection.notifyCloseIfNecessary(); this.idleConnections.addLast(connection); if (connection.equals(lastIdle)) { // all of the idle connections are closed. connection = this.idleConnections.poll(); break; } connection = null; } } else { break; } } return connection; }
这个方法看上去平平无奇,就是从缓存中获取一个connection的代理,如果connection没有打开,则直接关闭这个connection,并重新创建一个新的connection ,connection己经创建好了,但需要注意的是,返回给系统的connection是一个代理,这就是需要注意的第二点,那在这个代理类中帮我们做了哪些事情呢?意图是什么呢?下面着重看ChannelCachingConnectionProxy的close方法。
public void close() { if (CachingConnectionFactory.this.cacheMode == CacheMode.CONNECTION) { synchronized (CachingConnectionFactory.this.connectionMonitor) { if (!CachingConnectionFactory.this.idleConnections.contains(this)) { //如果connection没有open或者open的connection数量超过设置的connectionCacheSize值,则直接close connection if (!this.target.isOpen() || countOpenIdleConnections() >= CachingConnectionFactory.this.connectionCacheSize) { if (logger.isDebugEnabled()) { logger.debug("Completely closing connection '" + this + "'"); } destroy(); } if (logger.isDebugEnabled()) { logger.debug("Returning connection '" + this + "' to cache"); } //否则将connection存储到缓存中 CachingConnectionFactory.this.idleConnections.add(this); if (CachingConnectionFactory.this.connectionHighWaterMark .get() < CachingConnectionFactory.this.idleConnections.size()) { //设置connection的峰值 CachingConnectionFactory.this.connectionHighWaterMark .set(CachingConnectionFactory.this.idleConnections.size()); } CachingConnectionFactory.this.connectionMonitor.notifyAll(); } } } }
当看到代理的close()方法时,终于知道cacheMode模式为CONNECTION时,connection的创建及close逻辑,终于理解了之前提到的【每一次调用createConnection()方法都会新建一个或者从缓存中获取,根据你设置的ConnectionCacheSize的大小,当小于的时候会采用新建的策略,当大于等于的时候会采用从缓存中获取的策略】这句话的含义了。
接下来,我们再来看看createChannel()方法的实现。
public Channel createChannel(boolean transactional) { return getChannel(this, transactional); } private Channel getChannel(ChannelCachingConnectionProxy connection, boolean transactional) { //大于0的情况下才会通过 Semaphore 限制当前连接下可用的信道数量 if (this.channelCheckoutTimeout > 0) { Semaphore checkoutPermits = this.checkoutPermits.get(connection); if (checkoutPermits != null) { try { //尝试获得令牌,在超时时间内循环尝试获取,直到尝试获取成功或超时返回,不阻塞线程。 if (!checkoutPermits.tryAcquire(this.channelCheckoutTimeout, TimeUnit.MILLISECONDS)) { throw new AmqpTimeoutException("No available channels"); } if (logger.isDebugEnabled()) { logger.debug( "Acquired permit for " + connection + ", remaining:" + checkoutPermits.availablePermits()); } } catch (InterruptedException e) { Thread.currentThread().interrupt(); throw new AmqpTimeoutException("Interrupted while acquiring a channel", e); } } else { throw new IllegalStateException("No permits map entry for " + connection); } } LinkedList<ChannelProxy> channelList; if (this.cacheMode == CacheMode.CHANNEL) { channelList = transactional ? this.cachedChannelsTransactional : this.cachedChannelsNonTransactional; } else { channelList = transactional ? this.allocatedConnectionTransactionalChannels.get(connection) : this.allocatedConnectionNonTransactionalChannels.get(connection); } if (channelList == null) { throw new IllegalStateException("No channel list for connection " + connection); } ChannelProxy channel = null; //如果连接打开,获取连接中的可用信道 if (connection.isOpen()) { synchronized (channelList) { while (!channelList.isEmpty()) { channel = channelList.removeFirst(); if (logger.isTraceEnabled()) { logger.trace(channel + " retrieved from cache"); } if (channel.isOpen()) { break; } else { try { Channel target = channel.getTargetChannel(); if (target != null) { target.close(); } } catch (AlreadyClosedException e) { if (logger.isTraceEnabled()) { logger.trace(channel + " is already closed"); } } catch (IOException e) { if (logger.isDebugEnabled()) { logger.debug("Unexpected Exception closing channel " + e.getMessage()); } } catch (TimeoutException e) { if (logger.isWarnEnabled()) { logger.warn("TimeoutException closing channel " + e.getMessage()); } } channel = null; } } } if (channel != null) { if (logger.isTraceEnabled()) { logger.trace("Found cached Rabbit Channel: " + channel.toString()); } } } if (channel == null) { //如果信道为空,创建新的信道 channel = getCachedChannelProxy(connection, channelList, transactional); } return channel; }
上述代码的原理还是很简单的,从缓存中获取链接对应的信道列表,再从信道列表中获取第0个信道,如果信道不为空且isOpen,则返回给调用方,如果为空,则创建新的信道。但是有一个需要注意点,如果用户配置了channelCheckoutTimeout>0,这个时候用到了semaphore,那情况有所不同。我们先来看一个semaphore的场景。
用semaphore 实现租车用车的需求。
每个停车场入口都有一个提示牌,上面显示着停车场可用车辆可空闲车位数。为了保证每辆车都有车位可停,车辆数不能超过车位数。
- 当停车场有空闲车位时,提示租车平台去买辆新车,新车占一个停车位,这个时候你就可以租到刚刚创建的新车了。
- 租完车,你需要将车还到车位上,供其他人使用。
- 当异常原因 ,车被撞坏了,车位空闲,租车平台加一辆新车,放到空闲的车位上,等待的用户可以借到刚刚创建的新车。
- 当25辆车都被租出去了,只能等待有车被撞坏或者有人归还用车,才能租到车。
- 也就是说租车平台停车场有25个车位,只要有空闲车位,当新的用户来时,都可以去创建一辆新车,并租给用户,当车被租出去了,但是停车场车位空闲,这个时候不允许创建新车,因为租出去的车如果用户要归还,这个时候就没有车位可停了,因此,有25个空闲车位,最终在用的车只能是25辆。除非有车辆坏了,不能使用,才会去创建新车。getChannel的逻辑,就是要用户租车需要。
但是实际的getChannel()的逻辑和现实生活中还是有出入的,实际是停车场新建立,如果来100个人租车,先创建100辆新车,当用户用完来还车时,只有25个车位,先归的车占有车位,当第26辆车来归还时,没有车位了,直接将车给销毁,才是继续上面的逻辑,最终停车场就只有25辆在租的车辆。
有了上述业务场景,我们再来理解 checkoutPermits.tryAcquire(this.channelCheckoutTimeout, TimeUnit.MILLISECONDS)
这一行代码,应该就好理解了,在之前的代码中,我们默认设置了一个连接最多能拥有25个信道,如果当前25个信道都被占用了,那么就需要等待一定的时间【时间为channelCheckoutTimeout】,在等待的时间内看有没有信道被关闭,如25个关闭了一个变成了24个,或者有资源释放了信道,有一个空闲信道,这个时候tryAcquire()方法就会返回true,就开始从channelList中获取信道或者创建新的信道,否则抛出throw new AmqpTimeoutException(“No available channels”);异常。接下来,我们来看看,当信道不存在时,创建新的信道逻辑。
private ChannelProxy getCachedChannelProxy(ChannelCachingConnectionProxy connection, LinkedList<ChannelProxy> channelList, boolean transactional) { //通过Connection中delegate创建Channel对象 Channel targetChannel = createBareChannel(connection, transactional); if (logger.isDebugEnabled()) { logger.debug("Creating cached Rabbit Channel from " + targetChannel); } //向所有 ChannelListener 发布 onCreate 事件 getChannelListener().onCreate(targetChannel, transactional); Class<?>[] interfaces; if (this.publisherConfirms || this.publisherReturns) { interfaces = new Class<?>[] { ChannelProxy.class, PublisherCallbackChannel.class }; } else { interfaces = new Class<?>[] { ChannelProxy.class }; } //通过 Proxy.newProxyInstance创建一个实现了ChannelProxy接口的动态代理对象。 //所有对该实例的方法调用都会转交给CachedChannelInvocationHandler 的 invoke 方法处理 return (ChannelProxy) Proxy.newProxyInstance(ChannelProxy.class.getClassLoader(), interfaces, new CachedChannelInvocationHandler(connection, targetChannel, channelList, transactional)); }
先来看创建信道的过程。
private Channel createBareChannel(ChannelCachingConnectionProxy connection, boolean transactional) { if (this.cacheMode == CacheMode.CHANNEL) { if (!this.connection.isOpen()) { synchronized (this.connectionMonitor) { if (!this.connection.isOpen()) { this.connection.notifyCloseIfNecessary(); } if (!this.connection.isOpen()) { this.connection.target = null; createConnection(); } } } return doCreateBareChannel(this.connection, transactional); } else if (this.cacheMode == CacheMode.CONNECTION) { if (!connection.isOpen()) { synchronized (this.connectionMonitor) { this.allocatedConnectionNonTransactionalChannels.get(connection).clear(); this.allocatedConnectionTransactionalChannels.get(connection).clear(); connection.notifyCloseIfNecessary(); refreshProxyConnection(connection); } } return doCreateBareChannel(connection, transactional); } return null; }
在创建信道的过程中,根据不同的模式从不同的缓存中取出连接,再较验connection是否open,如果connection为空或者isOpen为false,则创建新的connection,从效果上来看,主要对connection的获取及较验,并保证connection可用。下面我们来看真正的创建信道的代码。
private Channel doCreateBareChannel(ChannelCachingConnectionProxy connection, boolean transactional) { Channel channel = connection.createBareChannel(transactional); //如果connectionFactory配置了 publisher-confirms="true",则调用confirmSelect方法 if (this.publisherConfirms) { try { channel.confirmSelect(); } catch (IOException e) { logger.error("Could not configure the channel to receive publisher confirms", e); } } if (this.publisherConfirms || this.publisherReturns) { if (!(channel instanceof PublisherCallbackChannelImpl)) { channel = new PublisherCallbackChannelImpl(channel); } } if (channel != null) { channel.addShutdownListener(this); } return channel; } public Channel createChannel(boolean transactional) { try { Channel channel = this.delegate.createChannel(); //如果开启事务【channel-transacted="true"】,则调用信道的txSelect方法 if (transactional) { channel.txSelect(); } return channel; } catch (IOException e) { throw RabbitExceptionTranslator.convertRabbitAccessException(e); } }
从getCachedChannelProxy方法中得知,最终通过JDK动态代理创建了信道的代理类,那么这个代理类帮我们做了哪些事情呢?我们进入代理类看看。
CachedChannelInvocationHandler.java
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable { String methodName = method.getName(); //当如果没有配置事务,却调用channel的txSelect方法,则抛出异常 if (methodName.equals("txSelect") && !this.transactional) { throw new UnsupportedOperationException("Cannot start transaction on non-transactional channel"); } if (methodName.equals("equals")) { return (proxy == args[0]); } else if (methodName.equals("hashCode")) { return System.identityHashCode(proxy); } else if (methodName.equals("toString")) { return "Cached Rabbit Channel: " + this.target + ", conn: " + this.theConnection; } else if (methodName.equals("close")) { if (CachingConnectionFactory.this.active) { synchronized (this.channelList) { if (!RabbitUtils.isPhysicalCloseRequired() && (this.channelList.size() < getChannelCacheSize() || this.channelList.contains(proxy))) { //释放信号量,比如25个车位,释放车位 releasePermitIfNecessary(proxy); //假的关闭信道,实际上是保存到缓存中 logicalClose((ChannelProxy) proxy); return null; } } } //真正的关闭信道 physicalClose(); //释放车位 releasePermitIfNecessary(proxy); return null; } else if (methodName.equals("getTargetChannel")) { return this.target; } else if (methodName.equals("isOpen")) { return this.target != null && this.target.isOpen(); } else if (methodName.equals("isTransactional")) { return this.transactional; } try { if (this.target == null || !this.target.isOpen()) { if (this.target instanceof PublisherCallbackChannel) { this.target.close(); throw new InvocationTargetException(new AmqpException("PublisherCallbackChannel is closed")); } this.target = null; } synchronized (this.targetMonitor) { if (this.target == null) { this.target = createBareChannel(this.theConnection, this.transactional); } //如果是其他方法,则直接通过反射调用即可 return method.invoke(this.target, args); } } catch (InvocationTargetException ex) { if (this.target == null || !this.target.isOpen()) { if (logger.isDebugEnabled()) { logger.debug("Detected closed channel on exception. Re-initializing: " + this.target); } this.target = null; synchronized (this.targetMonitor) { if (this.target == null) { this.target = createBareChannel(this.theConnection, this.transactional); } } } throw ex.getTargetException(); } }
上述过程中需要注意的是调用channel的close()方法时,如果当前信道列表中还存在车位或者该归还的车曾经注册过车位【也就是占有过车位】,则进行逻辑关闭,告诉等待的人,有车可租,如果该车从来没有停进过车位,并且停车场的车位都被别的车辆占有,直接close()信道。那我们来看看逻辑关闭信道的代码逻辑。
private void logicalClose(ChannelProxy proxy) throws Exception { if (this.target == null) { return; } if (this.target != null && !this.target.isOpen()) { synchronized (this.targetMonitor) { //如果信道不为空,并且信道isOpen为false if (this.target != null && !this.target.isOpen()) { if (this.target instanceof PublisherCallbackChannel) { //关闭信道 this.target.close(); } //如果列表中存在该信道,则将信道从列表中移除 if (this.channelList.contains(proxy)) { this.channelList.remove(proxy); } this.target = null; return; } } } //如果缓存列表中没有该信道,则将信道加到缓存列表尾部 if (!this.channelList.contains(proxy)) { if (logger.isTraceEnabled()) { logger.trace("Returning cached Channel: " + this.target); } this.channelList.addLast(proxy); setHighWaterMark(); } }
上述代码逻辑,如果channel isOpen等于false,直接关闭信道,如果信道列表中有该信道,直接将信道从信道列表中移除,如果信道是open的,并且信道缓存列表中没有该信道,将信道加到缓存未尾。
为了测试cacheMode的cache-mode="CONNECTION"模式,我们来模似Spring源码中的execute方法实现。
public void send(final String exchange, final String routingKey, final Message message, final CorrelationData correlationData) throws AmqpException { execute(new ChannelCallback<Object>() { @Override public Object doInRabbit(Channel channel) throws Exception { doSend(channel, exchange, routingKey, message, RabbitTemplate.this.returnCallback != null && RabbitTemplate.this.mandatoryExpression.getValue( RabbitTemplate.this.evaluationContext, message, Boolean.class), correlationData); return null; } }, obtainTargetConnectionFactoryIfNecessary(this.sendConnectionFactorySelectorExpression, message)); }
为什么要模似这个方法呢?因为execute方法内部帮我们做了很多,其中包括connection的关闭和channel的关闭【方法内部这行代码就是connection和channel的关闭代码ConnectionFactoryUtils.releaseResources(resourceHolder);】,所以我们要模似execute方法,来手动执行关闭channel和connection的操作。从而体现测试效果。
- 创建Spring配置文件
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:rabbit="http://www.springframework.org/schema/rabbit" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd http://www.springframework.org/schema/rabbit http://www.springframework.org/schema/rabbit/spring-rabbit-1.6.xsd"> <rabbit:connection-factory id="connectionFactory" host="172.16.157.242" username="guest" password="guest" port="5672" virtual-host="/" cache-mode="CONNECTION" /> <!--申明一个消息队列Queue durable:是否持久化 exclusive: 仅创建者可以使用的私有队列,断开后自动删除 auto_delete: 当所有消费客户端连接断开后,是否自动删除队列--> <rabbit:queue id="test_queue_key" name="test_queue_key" durable="true" auto-delete="false" exclusive="false" /> <!-- 声明Exchange rabbit:direct-exchange:定义exchange模式为direct --> <rabbit:direct-exchange name="test-mq-exchange" durable="true" auto-delete="false" id="test-mq-exchange"> <rabbit:bindings> <!--key 表示路由键--> <rabbit:binding queue="test_queue_key" key="test_queue_key"/> </rabbit:bindings> </rabbit:direct-exchange> <!-- spring template声明--> <rabbit:template id="rabbitTemplate" exchange="test-mq-exchange" connection-factory="connectionFactory" message-converter="jsonMessageConverter" /> <!-- 消息对象json转换类 --> <bean id="jsonMessageConverter" class="org.springframework.amqp.support.converter.Jackson2JsonMessageConverter"/> </beans>
- 开始测试
@Test public void test2() throws Exception { ApplicationContext context = new ClassPathXmlApplicationContext("classpath:spring_1_100/config_71_80/spring78_rabbitmq/spring8/spring78_mq_sender8.xml"); RabbitTemplate rabbitTemplate = (RabbitTemplate) context.getBean("rabbitTemplate"); Map<String, Object> msg = new HashMap(); msg.put("data", "hello,rabbmitmq!"); msg.put("time", System.currentTimeMillis()); org.springframework.expression.Expression expression = InvokeUtils.getFieldValue(rabbitTemplate, "sendConnectionFactorySelectorExpression"); String exchange = InvokeUtils.getFieldValue(rabbitTemplate, "exchange"); Message message = (Message) InvokeUtils.invokeMethod(rabbitTemplate, "convertMessageIfNecessary", msg); System.out.println(JSON.toJSONString(message)); ConnectionFactory connectionFactory = (ConnectionFactory) InvokeUtils.invokeMethod(rabbitTemplate, "obtainTargetConnectionFactoryIfNecessary", expression, message); System.out.println("connectionFactory = " + connectionFactory); CachingConnectionFactory cachingConnectionFactory = (CachingConnectionFactory)context.getBean("connectionFactory"); RabbitResourceHolder resourceHolder1 = ConnectionFactoryUtils.getTransactionalResourceHolder( (connectionFactory != null ? connectionFactory : cachingConnectionFactory), false); Channel channel1 = resourceHolder1.getChannel(); ChannelProxy channelProxy1 = (ChannelProxy) channel1; Channel targetChannel1 = channelProxy1.getTargetChannel(); InvokeUtils.invokeMethod(rabbitTemplate, "doSend", channel1, exchange, queue_key, message, true, null); RabbitResourceHolder resourceHolder2 = ConnectionFactoryUtils.getTransactionalResourceHolder( (connectionFactory != null ? connectionFactory : cachingConnectionFactory), false); Channel channel2 = resourceHolder2.getChannel(); ChannelProxy channelProxy2 = (ChannelProxy) channel2; Channel targetChannel2 = channelProxy2.getTargetChannel(); InvokeUtils.invokeMethod(rabbitTemplate, "doSend", channel1, exchange, queue_key, message, true, null); System.out.println("channel1是否等于channl2 " + (targetChannel1 == targetChannel2)); channelProxy1.close(); resourceHolder1.getConnection().close(); channelProxy2.close(); resourceHolder2.getConnection().close(); }
【测试结果】
为什么会出现channel1是否等于channl2 false不等于channel2呢?因为在第一次通过invokeMethod调用doSend方法后并没有显示的关闭connection和channel,这个时候再次通过getTransactionalResourceHolder()方法获取RabbitResourceHolder后,再获得channel,因为第一次发送消息完毕后并没有关闭channel,此时getChannel()发现并没有可用的channel,而是创建一个新的channel,【正如去租车,车还被其他人租着,并没有归还,此时租车平台只能给你创建一辆新车,当然停车位足够的情况下】,因此channel1 不等于channel2。那我们再来看另外一个例子,修改connection和channel关闭的位置,每次发送完消息后,即关闭connection和channel。
@Test public void test3() throws Exception { ApplicationContext context = new ClassPathXmlApplicationContext("classpath:spring_1_100/config_71_80/spring78_rabbitmq/spring8/spring78_mq_sender8.xml"); RabbitTemplate rabbitTemplate = (RabbitTemplate) context.getBean("rabbitTemplate"); Map<String, Object> msg = new HashMap(); msg.put("data", "hello,rabbmitmq!"); msg.put("time", System.currentTimeMillis()); org.springframework.expression.Expression expression = InvokeUtils.getFieldValue(rabbitTemplate, "sendConnectionFactorySelectorExpression"); String exchange = InvokeUtils.getFieldValue(rabbitTemplate, "exchange"); Message message = (Message) InvokeUtils.invokeMethod(rabbitTemplate, "convertMessageIfNecessary", msg); System.out.println(JSON.toJSONString(message)); ConnectionFactory connectionFactory = (ConnectionFactory) InvokeUtils.invokeMethod(rabbitTemplate, "obtainTargetConnectionFactoryIfNecessary", expression, message); System.out.println("connectionFactory = " + connectionFactory); CachingConnectionFactory cachingConnectionFactory = (CachingConnectionFactory)context.getBean("connectionFactory"); RabbitResourceHolder resourceHolder1 = ConnectionFactoryUtils.getTransactionalResourceHolder( (connectionFactory != null ? connectionFactory : cachingConnectionFactory), false); Channel channel1 = resourceHolder1.getChannel(); ChannelProxy channelProxy1 = (ChannelProxy) channel1; Channel targetChannel1 = channelProxy1.getTargetChannel(); InvokeUtils.invokeMethod(rabbitTemplate, "doSend", channel1, exchange, queue_key, message, true, null); channel1.close(); Connection connection1 = resourceHolder1.getConnection(); connection1.close(); RabbitResourceHolder resourceHolder2 = ConnectionFactoryUtils.getTransactionalResourceHolder( (connectionFactory != null ? connectionFactory : cachingConnectionFactory), false); Channel channel2 = resourceHolder2.getChannel(); ChannelProxy channelProxy2 = (ChannelProxy) channel2; Channel targetChannel2 = channelProxy2.getTargetChannel(); InvokeUtils.invokeMethod(rabbitTemplate, "doSend", channel1, exchange, queue_key, message, true, null); System.out.println("channel1是否等于channl2 " + (targetChannel1 == targetChannel2)); channel2.close(); resourceHolder2.getConnection().close(); }
【测试结果】
test2()和test3()方法的唯一区别就是将connection和channel关闭的位置不一样。test2()是先doSend1()方法后,再doSend2()方法,最后close connection1和channel1,test3()方法是先doSend1()方法后,再close connection1和channel2,再doSend2()方法,最后再close connection2和channel2方法。因此在close connection1和channel1之后,这个时候connection1和channel1被缓存起来,再次getConnection和getChannel时,实际上是从上次使用过没有关闭的connection和channel中获得,因此channel1等于channel2。在理解了connection和channel的获取逻辑以后,我们再来看doSend方法的内部逻辑。
protected void doSend(Channel channel, String exchange, String routingKey, Message message, boolean mandatory, CorrelationData correlationData) throws Exception { if (exchange == null) { // try to send to configured exchange exchange = this.exchange; } if (routingKey == null) { // try to send to configured routing key routingKey = this.routingKey; } if (logger.isDebugEnabled()) { logger.debug("Publishing message on exchange [" + exchange + "], routingKey = [" + routingKey + "]"); } setupConfirm(channel, correlationData); Message messageToUse = message; MessageProperties messageProperties = messageToUse.getMessageProperties(); if (mandatory) { messageProperties.getHeaders().put("spring_listener_return_correlation", this.uuid); } if (this.beforePublishPostProcessors != null) { for (MessagePostProcessor processor : this.beforePublishPostProcessors) { messageToUse = processor.postProcessMessage(messageToUse); } } if (this.userIdExpression != null && messageProperties.getUserId() == null) { String userId = this.userIdExpression.getValue(this.evaluationContext, messageToUse, String.class); if (userId != null) { messageProperties.setUserId(userId); } } BasicProperties convertedMessageProperties = this.messagePropertiesConverter .fromMessageProperties(messageProperties, this.encoding); channel.basicPublish(exchange, routingKey, mandatory, convertedMessageProperties, messageToUse.getBody()); //如果开启事务 if (isChannelLocallyTransacted(channel)) { //则调用channel.txCommit();提交事务 RabbitUtils.commitIfNecessary(channel); } }
发消息最关键的代码当然是basicPublish方法了,但是我们要讲isChannelLocallyTransacted方法和commitIfNecessary方法。
public static void commitIfNecessary(Channel channel) { Assert.notNull(channel, "Channel must not be null"); try { channel.txCommit(); } catch (IOException ex) { throw new AmqpIOException(ex); } }
如果配置了事务,channel肯定调用了txSelect方法,当发送消息没有抛出异常时,则调用txCommit()方法提交事务,如果抛出异常,则调用rollbackIfNecessary方法回滚事务。
public static void rollbackIfNecessary(Channel channel) { Assert.notNull(channel, "Channel must not be null"); try { channel.txRollback(); } catch (IOException ex) { throw new AmqpIOException(ex); } }
为了模似调用doInRabbit方法抛出异常,我们写了一个例子。
- 创建配置文件
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context" xmlns:rabbit="http://www.springframework.org/schema/rabbit" xmlns:tx="http://www.springframework.org/schema/tx" xsi:schemaLocation="http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-2.5.xsd http://www.springframework.org/schema/context https://www.springframework.org/schema/context/spring-context-2.5.xsd http://www.springframework.org/schema/rabbit http://www.springframework.org/schema/rabbit/spring-rabbit.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd"> <rabbit:connection-factory id="connectionFactory" host="172.16.157.242" username="guest" password="guest" port="5672" virtual-host="/"/> <!--申明一个消息队列Queue durable:是否持久化 exclusive: 仅创建者可以使用的私有队列,断开后自动删除 auto_delete: 当所有消费客户端连接断开后,是否自动删除队列--> <rabbit:queue id="test_queue_key" name="test_queue_key" durable="true" auto-delete="false" exclusive="false"/> <!-- 声明Exchange rabbit:direct-exchange:定义exchange模式为direct --> <rabbit:direct-exchange name="test-mq-exchange" durable="true" auto-delete="false" id="test-mq-exchange"> <rabbit:bindings> <!--key 表示路由键--> <rabbit:binding queue="test_queue_key" key="test_queue_key"/> </rabbit:bindings> </rabbit:direct-exchange> <!-- spring template声明--> <rabbit:template id="rabbitTemplate" exchange="test-mq-exchange" connection-factory="connectionFactory" message-converter="jsonMessageConverter" channel-transacted="true" /> <!-- 消息对象json转换类 --> <bean id="jsonMessageConverter" class="org.springframework.amqp.support.converter.Jackson2JsonMessageConverter"/> </beans>
- 开始测试
@Test public void test() throws Exception { ApplicationContext context = new ClassPathXmlApplicationContext("classpath:spring_1_100/config_71_80/spring78_rabbitmq/spring5/spring78_mq_sender5.xml"); RabbitTemplate rabbitTemplate = (RabbitTemplate) context.getBean("rabbitTemplate"); Map<String, Object> msg = new HashMap(); msg.put("data", "hello,rabbmitmq!"); msg.put("time", System.currentTimeMillis()); org.springframework.expression.Expression expression = InvokeUtils.getFieldValue(rabbitTemplate, "sendConnectionFactorySelectorExpression"); Message message = (Message) InvokeUtils.invokeMethod(rabbitTemplate, "convertMessageIfNecessary", msg); System.out.println(JSON.toJSONString(message)); ConnectionFactory connectionFactory = (ConnectionFactory) InvokeUtils.invokeMethod(rabbitTemplate, "obtainTargetConnectionFactoryIfNecessary", expression, message); ChannelCallback channelCallback = new ChannelCallback<Object>() { @Override public Object doInRabbit(Channel channel) throws Exception { String exchange = InvokeUtils.getFieldValue(rabbitTemplate, "exchange"); String routingKey = queue_key; InvokeUtils.invokeMethod(rabbitTemplate, "setupConfirm", channel, null); Message messageToUse = message; MessageProperties messageProperties = messageToUse.getMessageProperties(); if (false) { messageProperties.getHeaders().put(PublisherCallbackChannel.RETURN_CORRELATION_KEY, InvokeUtils.getFieldValue(rabbitTemplate, "uuid")); } Collection<MessagePostProcessor> beforePublishPostProcessors = InvokeUtils.getFieldValue(rabbitTemplate, "beforePublishPostProcessors"); if (beforePublishPostProcessors != null) { for (MessagePostProcessor processor : beforePublishPostProcessors) { messageToUse = processor.postProcessMessage(messageToUse); } } Expression userIdExpression = InvokeUtils.getFieldValue(rabbitTemplate, "userIdExpression"); if (userIdExpression != null && messageProperties.getUserId() == null) { StandardEvaluationContext evaluationContext = InvokeUtils.getFieldValue(rabbitTemplate, "evaluationContext"); String userId = userIdExpression.getValue(evaluationContext, messageToUse, String.class); if (userId != null) { messageProperties.setUserId(userId); } } MessagePropertiesConverter messagePropertiesConverter = InvokeUtils.getFieldValue(rabbitTemplate, "messagePropertiesConverter"); AMQP.BasicProperties convertedMessageProperties = messagePropertiesConverter. fromMessageProperties(messageProperties, InvokeUtils.getFieldValue(rabbitTemplate, "encoding")); channel.basicPublish(exchange, routingKey, false, convertedMessageProperties, messageToUse.getBody()); int i = 0; int j = 1; int c = j / i; // Check if commit needed boolean flag = (boolean) InvokeUtils.invokeMethod(rabbitTemplate, "isChannelLocallyTransacted", channel); if (flag) { // Transacted channel created by this template -> commit. RabbitUtils.commitIfNecessary(channel); } return null; } }; System.out.println("connectionFactory = " + connectionFactory); System.out.println("channelCallback=" + channelCallback); InvokeUtils.invokeMethod(rabbitTemplate, "execute", channelCallback, connectionFactory); }
上述方法完全模似doSend方法调用,因为有很多的方法或属性都是私有的,因此,你看到的都是反射调用方法或获取属性。先我们注入掉int c = j / i;这一行代码。测试
上面是正常逻辑,生产者生产消息,没有抛出异常的情况,消费者能顺利的接收到消息。当开启int c = j / i;这一行代码。开始测试
生产者发布了消息,但是没有提交成功
从上面测试结果来看,消费者没有收到新的消息,依然是第一次接收到的那条消息,那么如果我们设置channel-transacted=“false”, 并且开启int c = int c = j / 0;的代码会怎样呢?我们再来测试一把。
如果读者从头看到现在,我相信你对测试结果没有什么议意了,因为配置了channel-transacted=“true”,在创建channel时调用了channel的txSelect()方法,而发送消息时抛出异常,那么方法没法提交,进而调用了txRollback()方法,所以回滚了,所以消费者没有接收到消息,而channel-transacted="false"时,根本没有事务一说,只要调用basicPublish没有抛出异常,并且交换器,路由键,队列都存在的情况下,消息肯定是发送出去的。了解了channel-transacted的使用及原理,感觉用处还不是很大,那在Spring中该如何更加优雅的使用事务这一块呢?能不能像数据库事务一样,在service发布消息,如果service执行没有异常,则将rabbit消息发送出去,如果有异常则消息中断,不再发送。答案告诉你,当然是有的,那我们来看一个例子。
- 创建生产者配置文件
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context" xmlns:rabbit="http://www.springframework.org/schema/rabbit" xmlns:tx="http://www.springframework.org/schema/tx" xsi:schemaLocation="http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans-2.5.xsd http://www.springframework.org/schema/context https://www.springframework.org/schema/context/spring-context-2.5.xsd http://www.springframework.org/schema/rabbit http://www.springframework.org/schema/rabbit/spring-rabbit.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd"> <context:annotation-config/> <context:component-scan base-package="com.spring_1_100.test_71_80.test78_spring_rabbitmq.spring6_tx_transaction"></context:component-scan> <tx:annotation-driven transaction-manager="rabbitTransactionManager"></tx:annotation-driven> <rabbit:connection-factory id="connectionFactory" host="172.16.157.242" username="guest" password="guest" port="5672" virtual-host="/"/> <!--申明一个消息队列Queue durable:是否持久化 exclusive: 仅创建者可以使用的私有队列,断开后自动删除 auto_delete: 当所有消费客户端连接断开后,是否自动删除队列--> <rabbit:queue id="test_queue_key" name="test_queue_key" durable="true" auto-delete="false" exclusive="false"/> <!-- 声明Exchange rabbit:direct-exchange:定义exchange模式为direct --> <rabbit:direct-exchange name="test-mq-exchange" durable="true" auto-delete="false" id="test-mq-exchange"> <rabbit:bindings> <!--key 表示路由键--> <rabbit:binding queue="test_queue_key" key="test_queue_key"/> </rabbit:bindings> </rabbit:direct-exchange> <!-- spring template声明--> <rabbit:template id="rabbitTemplate" exchange="test-mq-exchange" connection-factory="connectionFactory" message-converter="jsonMessageConverter" channel-transacted="true"/> <--配置rabbit事务管理器--> <bean id="rabbitTransactionManager" class="org.springframework.amqp.rabbit.transaction.RabbitTransactionManager"> <property name="connectionFactory" ref="connectionFactory"></property> </bean> <!-- 消息对象json转换类 --> <bean id="jsonMessageConverter" class="org.springframework.amqp.support.converter.Jackson2JsonMessageConverter"/> </beans>
- 创建service层
@Service("userService") public class TransactionSevice { @Autowired private RabbitTemplate rabbitTemplate; public static final String queue_key = "test_queue_key"; @Transactional public void testTransaction(){ Map<String, Object> msg = new HashMap(); msg.put("data", "hello,rabbmitmq!"); msg.put("time", System.currentTimeMillis()); System.out.println("++++++++++++++++++++" + JSON.toJSONString(msg)); rabbitTemplate.convertAndSend(queue_key, msg); int i = 0; int j = 1; int c = j / i; } }
- 创建消费者配置文件
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:rabbit="http://www.springframework.org/schema/rabbit" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd http://www.springframework.org/schema/rabbit http://www.springframework.org/schema/rabbit/spring-rabbit-1.6.xsd"> <description>rabbitmq 连接服务配置</description> <!-- 连接配置 --> <rabbit:connection-factory id="connectionFactory" host="172.16.157.242" username="guest" password="guest" port="5672" virtual-host="/"/> <!--申明一个消息队列Queue durable:是否持久化 exclusive: 仅创建者可以使用的私有队列,断开后自动删除 auto_delete: 当所有消费客户端连接断开后,是否自动删除队列--> <rabbit:queue id="test_queue_key" name="test_queue_key" durable="true" auto-delete="false" exclusive="false"/> <bean id="queueListenter" class="com.spring_1_100.test_71_80.test78_spring_rabbitmq.spring0_product_consumer.QueueListenter"></bean> <rabbit:listener-container connection-factory="connectionFactory" acknowledge="auto" > <rabbit:listener queues="test_queue_key" ref="queueListenter" /> </rabbit:listener-container> </beans>
- 启动消费者
public class MQReciver { public static void main(String[] args) { new ClassPathXmlApplicationContext("classpath:spring_1_100/config_71_80/spring78_rabbitmq/spring6/spring78_mq_recive6.xml"); } }
- 生产消息
public class MQSender { public static void main(String[] args) { ApplicationContext context = new ClassPathXmlApplicationContext("classpath:spring_1_100/config_71_80/spring78_rabbitmq/spring6/spring78_mq_sender6.xml"); TransactionSevice transactionSevice = context.getBean(TransactionSevice.class); transactionSevice.testTransaction(); } }
依着之前的测试逻辑,还是先注释掉int c = j / i;的逻辑。开始测试。
生产者生产消息。
生产者生产消息没有抛出异常,消费者顺利接收到消息。
开启int c = j / 0;消费者不变。重新发送消息。
生产者生产消息抛出异常
但是消费者并没有接收到新的消息。
效果己经很明显了,但是大家要注意的一点是,这个测试例子中并没有使用channel-transacted=“true”,也实现了消息的事务功能,那Spring是如何帮我们实现的呢?如果开启了事务,当调用service方法时,会调用切面事务逻辑,在进入目标方法之前,会先调用RabbitTransactionManager的doBegin方法。关于事务切面这一块,在之前分析Spring事务己经分析得很很详细了,这里就不做过多的讲解。
protected void doBegin(Object transaction, TransactionDefinition definition) { if (definition.getIsolationLevel() != TransactionDefinition.ISOLATION_DEFAULT) { throw new InvalidIsolationLevelException("AMQP does not support an isolation level concept"); } RabbitTransactionObject txObject = (RabbitTransactionObject) transaction; RabbitResourceHolder resourceHolder = null; try { //第二个参数true,等同于配置文件中的channel-transacted=“true” resourceHolder = ConnectionFactoryUtils.getTransactionalResourceHolder(getConnectionFactory(), true); if (logger.isDebugEnabled()) { logger.debug("Created AMQP transaction on channel [" + resourceHolder.getChannel() + "]"); } txObject.setResourceHolder(resourceHolder); txObject.getResourceHolder().setSynchronizedWithTransaction(true); int timeout = determineTimeout(definition); if (timeout != TransactionDefinition.TIMEOUT_DEFAULT) { txObject.getResourceHolder().setTimeoutInSeconds(timeout); } //绑定事务资源 TransactionSynchronizationManager.bindResource(getConnectionFactory(), txObject.getResourceHolder()); } catch (AmqpException ex) { if (resourceHolder != null) { ConnectionFactoryUtils.releaseResources(resourceHolder); } throw new CannotCreateTransactionException("Could not create AMQP transaction", ex); } }
上述开启事务的过程中,需要注意的是getTransactionalResourceHolder()方法的第二个参数,这个参数等于同我们配置的channel-transacted=“true”,也就是读者可能会觉得,为什么我们没有配置channel-transacted=“true”,也有事务的效果。还有一个比较重要的逻辑是bindResource()绑定事务资源,进入这个方法内部看看。
public static void bindResource(Object key, Object value) throws IllegalStateException { Object actualKey = TransactionSynchronizationUtils.unwrapResourceIfNecessary(key); Assert.notNull(value, "Value must not be null"); Map<Object, Object> map = resources.get(); // set ThreadLocal Map if none found if (map == null) { map = new HashMap<Object, Object>(); resources.set(map); } Object oldValue = map.put(actualKey, value); // Transparently suppress a ResourceHolder that was marked as void... if (oldValue instanceof ResourceHolder && ((ResourceHolder) oldValue).isVoid()) { oldValue = null; } if (oldValue != null) { throw new IllegalStateException("Already value [" + oldValue + "] for key [" + actualKey + "] bound to thread [" + Thread.currentThread().getName() + "]"); } if (logger.isTraceEnabled()) { logger.trace("Bound value [" + value + "] for key [" + actualKey + "] to thread [" + Thread.currentThread().getName() + "]"); } }
这个方法也是平平无奇,就是将connectionFactory和resourceHolder存储到resource中,没有什么新意,但是其这么做的作用是什么呢?不知道读者有没有记得doSend方法内部的isChannelLocallyTransacted()这个方法,
... //发送消息 channel.basicPublish(exchange, routingKey, mandatory, convertedMessageProperties, messageToUse.getBody()); //判断有没有事务 if (isChannelLocallyTransacted(channel)) { //提交事务 RabbitUtils.commitIfNecessary(channel); }
isChannelLocallyTransacted这个方法很特别,之前我们只说了一半,就是判断这个方法是否有事务,但实际并不是如此。
protected boolean isChannelLocallyTransacted(Channel channel) { return isChannelTransacted() && !ConnectionFactoryUtils.isChannelTransactional(channel, getConnectionFactory()); }
isChannelLocallyTransacted这个方法的返回值由两个控制,如果想让isChannelLocallyTransacted返回true,必需配置channel-transacted=“true” ,并且isChannelTransactional为false才会执行提交事务的操作,我们看看isChannelTransactional()方法的内部实现。
public static boolean isChannelTransactional(Channel channel, ConnectionFactory connectionFactory) { if (channel == null || connectionFactory == null) { return false; } RabbitResourceHolder resourceHolder = (RabbitResourceHolder) TransactionSynchronizationManager .getResource(connectionFactory); return (resourceHolder != null && resourceHolder.containsChannel(channel)); }
显然,在doBegin方法内部,我们bindSource(),因此getSource()肯定不为空,正常情况下containsChannel方法都返回true,因此即使我们配置了channel-transacted=“true”,RabbitUtils.commitIfNecessary(channel);仍然不会执行,将交由外部RabbitTransactionManager执行提交和回滚操作。当抛出异常时将执行doRollback()方法回滚操作。
protected void doRollback(DefaultTransactionStatus status) { RabbitTransactionObject txObject = (RabbitTransactionObject) status.getTransaction(); RabbitResourceHolder resourceHolder = txObject.getResourceHolder(); resourceHolder.rollbackAll(); } public void rollbackAll() { for (Channel channel : this.channels) { if (logger.isDebugEnabled()) { logger.debug("Rolling back messages to channel: " + channel); } RabbitUtils.rollbackIfNecessary(channel); if (this.deliveryTags.containsKey(channel)) { for (Long deliveryTag : this.deliveryTags.get(channel)) { try { channel.basicReject(deliveryTag, true); } catch (IOException ex) { throw new AmqpIOException(ex); } } RabbitUtils.commitIfNecessary(channel); } } } public static void rollbackIfNecessary(Channel channel) { Assert.notNull(channel, "Channel must not be null"); try { channel.txRollback(); } catch (IOException ex) { throw new AmqpIOException(ex); } }
如果没有抛出异常,显然是执行doCommit方法。
protected void doCommit(DefaultTransactionStatus status) { RabbitTransactionObject txObject = (RabbitTransactionObject) status.getTransaction(); RabbitResourceHolder resourceHolder = txObject.getResourceHolder(); resourceHolder.commitAll(); } public void commitAll() throws AmqpException { try { for (Channel channel : this.channels) { if (this.deliveryTags.containsKey(channel)) { for (Long deliveryTag : this.deliveryTags.get(channel)) { channel.basicAck(deliveryTag, false); } } channel.txCommit(); } } catch (IOException e) { throw new AmqpException("failed to commit RabbitMQ transaction", e); } }
总结:
Spring整合RabbitMQ源码分析到这里,我相信大家己经对Spring如何封装Rabbit这一块己经有了深入的理解,如果还有什么疑问,请在博客下方留言,我看到一定来修整和完善博客。但是还有一些问题没有涉及到,比如如果使用注解的方式配置RabbitMQ,源码又是怎样的呢?这个问题,等我在分析Spring Boot源码时,再来分析这一块的代码了。
参考文章
RabbitMQ连接池
https://www.jianshu.com/p/2e90f9070995
Semaphore 使用及原理
https://zhuanlan.zhihu.com/p/98593407
本文所涉及到的源码的github地址是
https://github.com/quyixiao/spring_tiny/tree/master/src/main/java/com/spring_1_100/test_71_80/test78_spring_rabbitmq